Datadog Operations Skill Skill

Datadog Operations Skill

Comprehensive Datadog automation: query APM/logs/metrics/RUM/database, create monitors/dashboards/synthetics, manage incidents, trigger workflows, analyze costs and LLM usage.

Quick Setup

macOS

./setup.sh

Linux

./setup-linux.sh

Windows (PowerShell as Administrator)

.\setup-windows.ps1

Platform-Specific Setup

macOS

Install jq (required for JSON processing):

# Homebrew (recommended)
brew install jq

# MacPorts
sudo port install jq

# Direct binary (Apple Silicon)
curl -L -o ~/bin/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-macos-arm64
chmod +x ~/bin/jq

# Direct binary (Intel)
curl -L -o ~/bin/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-macos-amd64
chmod +x ~/bin/jq

Set environment variables in ~/.zshrc:

export DD_API_KEY="your_api_key"
export DD_APP_KEY="your_application_key"
export DD_SITE="datadoghq.com"

Linux

Install dependencies:

# Debian/Ubuntu
sudo apt-get install -y jq curl bc

# Fedora/RHEL
sudo dnf install -y jq curl bc

# Arch
sudo pacman -S jq curl bc

Set environment variables in ~/.bashrc:

export DD_API_KEY="your_api_key"
export DD_APP_KEY="your_application_key"
export DD_SITE="datadoghq.com"

Windows

Option 1: WSL (Recommended)

wsl --install
# Then run Linux setup inside WSL
./setup-linux.sh

Option 2: Git Bash Install Git for Windows, then run ./setup.sh in Git Bash.

Option 3: Native PowerShell

# Install jq
winget install jqlang.jq
# Or: choco install jq

# Set environment variables
[Environment]::SetEnvironmentVariable("DD_API_KEY", "your_key", "User")
[Environment]::SetEnvironmentVariable("DD_APP_KEY", "your_app_key", "User")

Use Python scripts directly: python python\script_name.py

Required Permissions

Get keys from Datadog → Organization Settings → API Keys / Application Keys

For workflow automation (trigger-workflow): Enable "Actions API Access" on your app key: Datadog → Organization Settings → Application Keys → Click key → Enable "Actions API Access"

Quick Reference

Investigation Scripts

| Script | Purpose | Example | |--------|---------|---------| | query-apm.sh | Find slow endpoints | bash scripts/query-apm.sh --service my-service --duration 1h | | search-logs.sh | Search logs for errors | bash scripts/search-logs.sh --query "status:error" --duration 1h | | query-security-signals.sh | Find security threats | bash scripts/query-security-signals.sh --severity critical | | query-watchdog.sh | Anomaly detection | bash scripts/query-watchdog.sh --service my-service | | query-metrics.sh | Fetch metrics data | bash scripts/query-metrics.sh --metric "system.cpu.user" | | analyze-usage-cost.sh | FinOps cost analysis | bash scripts/analyze-usage-cost.sh --duration 30d | | analyze-llm.sh | LLM observability | bash scripts/analyze-llm.sh --service my-llm-app | | query-slos.sh | SLO status | bash scripts/query-slos.sh --service payment-api | | query-service-catalog.sh | Service metadata | bash scripts/query-service-catalog.sh list | | query-database.sh | DB performance | bash scripts/query-database.sh --host postgres-prod | | query-rum.sh | Frontend performance | bash scripts/query-rum.sh --application abc-123 | | query-kubernetes.sh | K8s workloads | bash scripts/query-kubernetes.sh --cluster prod | | query-containers.sh | Container metrics | bash scripts/query-containers.sh --duration 1h | | query-network.sh | Network monitoring | bash scripts/query-network.sh --duration 1h |

Automation Scripts

| Script | Purpose | Example | |--------|---------|---------| | manage-monitors.sh | Create/mute monitors | bash scripts/manage-monitors.sh list | | create-dashboard.sh | Generate dashboards | bash scripts/create-dashboard.sh --service my-service | | trigger-workflow.sh | Execute workflows | bash scripts/trigger-workflow.sh list | | manage-incidents.sh | Incident management | bash scripts/manage-incidents.sh list | | manage-synthetics.sh | Synthetic tests | bash scripts/manage-synthetics.sh list | | manage-on-call.sh | On-call scheduling | bash scripts/manage-on-call.sh list | | manage-status-pages.sh | Status pages | bash scripts/manage-status-pages.sh list | | verify-setup.sh | Validate config | bash scripts/verify-setup.sh |

Workflows

Investigate Production Issue

bash scripts/query-watchdog.sh --service affected-service --duration 24h
bash scripts/query-apm.sh --service affected-service --duration 1h
bash scripts/search-logs.sh --service affected-service --status error --duration 1h

Check Cost & Usage

bash scripts/analyze-usage-cost.sh --duration 30d --product all
bash scripts/analyze-usage-cost.sh --duration 30d | jq '.recommendations[] | select(.priority == "high")'

Monitor LLM Application

bash scripts/analyze-llm.sh --service my-genai-app --duration 24h

Output Format

All scripts return structured JSON to stdout, status messages to stderr:

bash scripts/query-apm.sh --service my-service 2>/dev/null | jq '.summary'

Python Alternative

Python versions available in python/ directory:

python3 -m venv .venv && source .venv/bin/activate
pip install -r python/requirements.txt
python python/query_apm.py --service my-service --json

Go CLI Alternative

Single-binary CLI available in ../dd-skill-test-go/:

# Native on all platforms - no dependencies
datadog-cli apm --service my-service --duration 1h
datadog-cli logs --query "status:error"
datadog-cli monitors list
datadog-cli health

See ../dd-skill-test-go/README.md for installation.

Notes

Advanced Scripts: Some scripts require Python. Run ./setup.sh first.
Windows: Use WSL, Git Bash, or Python scripts directly.
Go CLI: Recommended for Windows - native binary, no dependencies.

Agent Skills: Datadog Operations Skill

Install this agent skill to your local

Skill Files