Config Scraper YAML Skill Skill

Config Scraper YAML Skill

Goal

Turn a user request into a valid ScrapeConfig YAML that can be applied in Mission Control.

How to Use

Identify the scraper type(s) the user needs (kubernetes, exec, http, file, sql, logs, aws, gcp, azure, githubActions, slack, trivy, terraform).
Ask only the minimum clarifying questions required to produce correct YAML (cluster/region, credentials source, namespace, schedule, filters, and output mapping).
Produce a single ScrapeConfig YAML in a fenced code block. Keep it minimal and runnable.
If the user mentions secrets, always use secret references (do not inline sensitive values).
If a request cannot be expressed in a scraper type, explain the limitation and provide the closest working YAML (often via exec).

Inputs Checklist

Target system and scraper type
Credentials source (secret name + key, or connection name)
Schedule (optional; omit if not specified)

Output Rules

Output YAML only, in a single code block.
Use apiVersion: configs.flanksource.com/v1 and kind: ScrapeConfig.
Set metadata.name to a short, unique slug.

Canonical Snippets

Kubernetes RBAC

RBAC extraction is automatic when these resources are watched — no transform needed:

apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
  name: k8s-rbac
spec:
  kubernetes:
    - clusterName: my-cluster
      watch:
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: ClusterRole
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: ClusterRoleBinding
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: Role
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
        - apiVersion: v1
          kind: ServiceAccount

Kubernetes Audit Logs (Loki)

apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
  name: k8s-audit-logs
spec:
  logs:
    - name: k8s-audit
      type: KubernetesAudit
      loki:
        url: http://loki.monitoring:3100
        query: '{job="kube-audit"}'
      fieldMapping:
        id: ["responseStatus.metadata.uid"]
        message: ["responseStatus.message"]
        timestamp: ["stageTimestamp", "requestReceivedTimestamp"]
        severity: ["responseStatus.code"]

AWS CloudTrail

apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
  name: aws-cloudtrail
spec:
  aws:
    - connection: connection://aws/credentials
      region:
        - us-east-1
      cloudtrail:
        maxAge: 7d
        exclude:
          - AssumeRole
          - DecodeAuthorizationMessage

GCP Audit Logs

apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
  name: gcp-audit
spec:
  gcp:
    - project: my-project
      connection: connection://gcp/credentials
      auditLogs:
        dataset: my-project.audit_dataset.cloudaudit_googleapis_com_activity
        since: 30d
        serviceNames:
          - compute.googleapis.com
          - iam.googleapis.com
        principalEmails:
          - "@my-org.com"

MSSQL Permissions (full: true)

See @skills/create-config-scraper/references/access-logs.md for the complete MSSQL example showing external_users, external_roles, and config_access extraction via CEL transforms.

apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
  name: mssql-scraper
spec:
  full: true
  sql:
    - type: MSSQL::Logon
      connection: connection://mssql/credentials
      id: $.id
      name: $.name
      transform:
        expr: |
          # CEL transform emitting config, external_users, external_roles, config_access
          # See access-logs.md reference for the full transform expression

Reference

Scraper Schema Map (Bundled)

Use the bundled per-scraper schemas below. Only open the schema for the requested scraper type(s).

Kubernetes: @skills/create-config-scraper/references/schemas/config_kubernetes.schema.json
Kubernetes file: @skills/create-config-scraper/references/schemas/config_kubernetesfile.schema.json
Exec: @skills/create-config-scraper/references/schemas/config_exec.schema.json
HTTP: @skills/create-config-scraper/references/schemas/config_http.schema.json
File: @skills/create-config-scraper/references/schemas/config_file.schema.json
SQL: @skills/create-config-scraper/references/schemas/config_sql.schema.json
Logs: @skills/create-config-scraper/references/schemas/config_logs.schema.json
AWS: @skills/create-config-scraper/references/schemas/config_aws.schema.json
GCP: @skills/create-config-scraper/references/schemas/config_gcp.schema.json
Azure: @skills/create-config-scraper/references/schemas/config_azure.schema.json
Azure DevOps: @skills/create-config-scraper/references/schemas/config_azuredevops.schema.json
GitHub Actions: @skills/create-config-scraper/references/schemas/config_githubactions.schema.json
Slack: @skills/create-config-scraper/references/schemas/config_slack.schema.json
Trivy: @skills/create-config-scraper/references/schemas/config_trivy.schema.json
Terraform: @skills/create-config-scraper/references/schemas/config_terraform.schema.json

Additional References

Access Logs & RBAC: @skills/create-config-scraper/references/access-logs.md
Config DB documentation: https://flanksource.com/docs/guide/config-db/llms.txt

Agent Skills: Config Scraper YAML Skill

Install this agent skill to your local

Skill Files