Agent Skills: Analyze CI Failures

Analyze failed GitHub Action jobs. Takes one or more GitHub URLs (job, workflow-run, or PR) and summarizes each failure with root cause and log paths.

ID: mlflow/mlflow/analyze-ci

Repository

mlflowLicense: Apache-2.0
28,6316,571

Install this agent skill to your local

npx skills add mlflow/mlflow --skill analyze-ci

Skill Files

The files that make up analyze-ci.

Download Skill

Loading file tree…

.claude/skills/analyze-ci/SKILL.md

Skill Metadata

Name
analyze-ci
Description
Analyze failed GitHub Action jobs. Takes one or more GitHub URLs (job, workflow-run, or PR) and summarizes each failure with root cause and log paths.

Analyze CI Failures

Fetch logs from failed GitHub Action jobs and produce a focused per-job failure summary.

URLs provided: $ARGUMENTS

If no URLs are listed above, ask for a job, run, or PR URL and stop. fetch-logs requires at least one and exits 2 without.

Prerequisites

  • GitHub Token: Auto-detected via gh auth token, or set GH_TOKEN.

Steps

  1. Fetch logs. Run, single-quoting each URL so the shell does not interpret ? or other special characters in it:

    uv run --package skills skills fetch-logs '<url>' ['<url>' ...]
    

    The command prints one block per failed job containing the workflow/job name, URL, failed step, and paths to the cached raw log, failed-step log, and optional package versions and snapshot changes files. Snapshot changes are extracted from the existing raw log, not fetched or compared separately.

  2. Read each failed-step log and summarize it. For every block, Read the file at its Failed step log: path, then identify:

    • The root cause.
    • Specific error messages (assertion errors, exceptions, stack traces).
    • Full pytest test names where applicable (e.g. tests/test_foo.py::test_bar).
    • A short log snippet showing the error context.

    If Snapshot changes: is present, read that file too and summarize relevant package changes alongside the failure, including the baseline run link when available. Changes use uv's Updated name vold -> vnew, Added name vversion, and Removed name vversion format. Treat changes as diagnostic evidence, not proof of the root cause. Report unavailable baselines or comparison errors as such, not as package removals. Do not fetch artifacts or perform another comparison. Older or non-cross-version logs may have no snapshot changes block; analyze them normally.

  3. Format each summary with these fields, then a blank line, then the 1-2 paragraph summary.

    • Failed job: <workflow name> / <job name>
    • Failed step: <step name>
    • URL: <job_url>
    • Raw log: <raw_log_path>
    • Failed step log: <failed_step_log_path>
    • Package versions: <package_versions_path> (if present)
    • Snapshot changes: <snapshot_changes_path> (if present)

    Preserve the Raw log:, Failed step log:, Package versions:, and Snapshot changes: paths verbatim from step 1 so downstream agents can grep deeper.

Invocation examples

# All failed jobs on a PR
/analyze-ci https://github.com/mlflow/mlflow/pull/19601

# All failed jobs in one workflow run
/analyze-ci https://github.com/mlflow/mlflow/actions/runs/22626454465

# Specific job by URL
/analyze-ci https://github.com/mlflow/mlflow/actions/runs/12345/job/67890

# Multiple URLs at once
/analyze-ci https://github.com/mlflow/mlflow/actions/runs/123/job/456 https://github.com/mlflow/mlflow/actions/runs/789/job/012

More agent skills