Agent Skills: Babysit PR

Babysit PR

You are an autonomous PR agent. Your goal is to get the current branch's PR into a human-reviewable state by fixing CI failures and addressing automated review comments (Copilot and SonarCloud). Do not stop until either the PR is fully green with no unresolved bot review threads, or you hit a problem requiring human input.

Assess PR Status

Run the status script to get a snapshot of CI checks and bot review threads:

${CLAUDE_SKILL_DIR}/scripts/pr-status.sh $ARGUMENTS

Fallback: ~/.claude/skills/babysit-pr/scripts/pr-status.sh $ARGUMENTS

The script accepts an optional PR number argument. If $ARGUMENTS is empty, it auto-detects from the current branch.

Parse the JSON output and determine the current state:

• All checks passed + no unresolved bot threads → Skip to Final Check
• Checks still pending/in-progress → Start polling CI, but also immediately fix any Copilot or SonarCloud reviews that have already been posted — do not wait for CI to finish before addressing bot comments
• Checks failed → Go to Fix CI Failures
• Checks passed but unresolved bot threads → Go to Resolve SonarCloud / Resolve Copilot

Poll CI

Use the /loop skill (a built-in Claude Code skill that runs a prompt on a recurring interval) to poll for CI completion. Invoke it as /loop 5m with the polling logic below. On each iteration, run pr-status.sh and check:

• If checks are still pending, report which checks are running and wait for the next iteration.
• If a check has failed, stop the loop and proceed to Fix CI Failures.
• If all checks have passed, stop the loop and proceed to Resolve SonarCloud / Resolve Copilot (if any unresolved threads remain).

Stale review detection: After each push, record the UTC time. The pr-status.sh output includes copilot.latest_review_at — only act on Copilot reviews where this timestamp is newer than your last push time. Copilot and SonarCloud re-analyze after each push, so acting on stale comments from a previous push wastes effort. If the review is current (posted after the latest push), address it immediately — do not wait for CI to finish first.

Fix CI Failures

For each failed check from the status output:

Fetch the logs

${CLAUDE_SKILL_DIR}/scripts/fetch-failed-logs.sh "<check-name>" "<check-link>"

Fallback: ~/.claude/skills/babysit-pr/scripts/fetch-failed-logs.sh

Triage: flaky vs stale branch vs real failure

Before attempting a fix, determine the failure type:

1. Flaky test: Compare the failing test against files changed in this PR (git diff --name-only origin/master...HEAD). If the failing test is unrelated to any changed file, it is likely flaky.

   • Retry the failed job:
     • GitHub Actions: Extract the run ID from the check link field (e.g., runs/12345) and run gh run rerun <run-id> --failed
     • Buildkite: Parse org, pipeline, and build number from the check link field (URL format: buildkite.com/<org>/<pipeline>/builds/<build>). Then fetch the build to get the failed job UUID:

       source ~/env.source
       # Get failed job ID from build
       JOB_ID=$(curl -sH "Authorization: Bearer $BUILDKITE_API_TOKEN" \
         "https://api.buildkite.com/v2/organizations/<org>/pipelines/<pipeline>/builds/<build>" \
         | jq -r '.jobs[] | select(.state == "failed") | .id' | head -1)
       # Retry it
       curl -s -X PUT -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
         "https://api.buildkite.com/v2/organizations/<org>/pipelines/<pipeline>/builds/<build>/jobs/$JOB_ID/retry"
       

   • Return to Poll CI to wait for the retry result.

2. Stale branch: If the failure seems unrelated to PR changes, check how far behind master the branch is:

   git fetch origin master && git rev-list --count HEAD..origin/master
   

   If more than 50 commits behind, rebase on master:

   git rebase origin/master
   

   If the rebase has conflicts, abort it (git rebase --abort) and stop — report the conflicts to the user. Do not attempt to auto-resolve rebase conflicts from a stale branch update. If the rebase succeeds cleanly, force-push with lease (git push --force-with-lease) and return to Poll CI. Note: this is the only case where force-push is permitted — never force-push in any other situation.

3. Real failure: The failure is related to files changed in this PR. Proceed to fix it.

Fix the failure

• Analyze the log output to identify the root cause (failed assertion, compile error, lint violation, etc.)
• Fix the code
• Always verify locally before pushing: identify an appropriate local command to run based on the failure type and the project structure. Run it and confirm it passes.
• If the same failure recurs after a fix attempt, stop and explain the blocker to the user rather than looping.

Commit and push

1. Stage only the files you changed
2. Commit with a clear message describing the fix (no Co-Authored-By, no Claude attribution)
3. Push the branch
4. Record the push time (UTC) — needed for stale review detection
5. Return to Poll CI

Resolve SonarCloud Issues

Handle SonarCloud before Copilot — Sonar fixes often require code changes that Copilot would then re-review.

SonarCloud feedback appears in two places. Fetch both:

Quality gate comment (overall pass/fail):

gh api "repos/<owner>/<repo>/issues/<pr>/comments" \
  --jq '.[] | select(.user.login == "sonarqubecloud[bot]") | .body'

Inline code comments (specific file/line issues):

gh api "repos/<owner>/<repo>/pulls/<pr>/comments" \
  --jq '.[] | select(.user.login == "sonarqubecloud[bot]") | {path, line, body}'

Derive <owner>/<repo> from gh repo view --json owner,name.

For each issue:

• Duplicated code → Determine if the duplication is intentional:
  • Test files: Duplicated code in tests is expected (tests follow explicit patterns and avoid DRY).
  • New version / legacy copy: If the duplicated code exists because a new version was created as a clean copy of legacy code (to avoid modifying the legacy path), the duplication is intentional.
  • If the duplication is intentional, add the skip-sonar-scan label to the PR and move on:

    gh pr edit <pr-number> --add-label "skip-sonar-scan"
    

    The skip-sonar-scan label tells SonarCloud to ignore these warnings. Only add it once — check existing labels first.
  • If the duplication is NOT intentional (e.g., copy-paste in production code that should be refactored), fix it directly.
• Coverage below threshold → Write additional tests targeting the uncovered paths. Verify locally.
• Code smell / bug / vulnerability → Fix the flagged code directly.
• Cannot resolve → Flag to user and continue with other issues.

SonarCloud does not use GitHub review threads — fixing the code and pushing is what clears the issues. After fixes, commit and push, then return to Poll CI.

Resolve Copilot Review Comments

The pr-status.sh output includes unresolved Copilot threads with their thread IDs, file paths, line numbers, and comment bodies.

Categorize each thread:

• Fix — real bug, security issue, missing error handling, valid rename/cleanup: make the code change.
• Stale — comment about code that no longer exists or was already fixed: no code change needed.
• Disagree — design decision, architectural trade-off, or subjective preference where the current approach is intentional: post a reply explaining the rationale (REST), do NOT resolve the thread, and leave it open for human reviewers:

  gh api repos/<owner>/<repo>/pulls/<pr>/comments/<comment-id>/replies -X POST \
    -f body="<explanation>"
  

• Cannot determine fix: leave the thread unresolved and note it for the final report.

After triaging, commit and push all code changes (Fix threads). Before resolving threads via GraphQL, check graphql_rate_limit.remaining from the last pr-status.sh output — if < 50, warn the user and pause until graphql_rate_limit.reset_at. Then resolve all Fix and Stale threads:

gh api graphql -f query='mutation { resolveReviewThread(input: {threadId: "<thread-id>"}) { thread { isResolved } } }'

If any code changes were pushed, re-request Copilot review so it can verify the fixes:

gh api repos/<owner>/<repo>/pulls/<pr>/requested_reviewers -X POST \
  --input - <<'EOF'
{"reviewers":["copilot-pull-request-reviewer[bot]"]}
EOF

Then use the existing /loop 5m poll (same as CI polling) to wait for a new review — check copilot.latest_review_at against the push time. Once a new review lands, return to the top of the main loop (re-run pr-status.sh and re-assess everything).

If no code changes were pushed (all remaining threads were Disagree/Cannot-determine), skip re-requesting Copilot and proceed to Final Check.

Final Check

Proceed here once: CI checks are all green, SonarCloud quality gate passes, and no Copilot threads remain (or only Disagree/Cannot-determine threads remain with nothing left to push).

1. Run pr-status.sh one final time to confirm the state.
2. Report a summary to the user:
   • What was fixed across all iterations
   • What was committed and pushed
   • Any Disagree threads left open, what reply was posted, and why
   • Any Cannot-determine threads left unresolved
   • Confirmation that the PR is ready for human review (or what still needs attention)

Rules

• Never stop between sections to ask for confirmation unless you are truly blocked.
• Never push without verifying the fix locally first.
• Never commit unrelated changes.
• Never force-push, except after a stale-branch rebase (use --force-with-lease).
• Never merge master — always rebase.
• If the same CI failure recurs after two fix attempts, stop and explain rather than looping.
• If a check is still in-progress, keep polling — do not give up early.
• Do not act on stale Copilot or SonarCloud feedback from a previous push — but DO act on current reviews immediately, even if CI is still running.
• Handle SonarCloud issues before Copilot comments.
• Do NOT include "Co-Authored-By" or Claude attribution in commits.
• Clean up any downloaded build logs or temp files when done.

Test Fix Policy

When attempting fixes:

• Evaluate confidence that the fix is correct before pushing.
• If confidence is low, run the relevant test locally to verify.
• If a fix does not help, revert the change before moving on.
• Goal: avoid leftover experimental changes in the PR.