Agent-Skills.md

Agent Skills: ds-fix

This skill should be used when the user asks to 'fix analysis', 'wrong results', 'notebook error', 'reviewer feedback', 'data changed', 'debug notebook', or needs mid-analysis course-correction for wrong results, notebook errors, or data changes.

UncategorizedID: edwinhu/workflows/ds-fix

Repository

edwinhu/workflows
edwinhu
5

Install this agent skill to your local

pnpm dlx add-skill https://github.com/edwinhu/workflows/tree/HEAD/skills/ds-fix

Skill Files

Browse the full folder contents for ds-fix.

Download Skill

Loading file tree…

skills/ds-fix/SKILL.md

Skill Metadata

Name
ds-fix
Description
"This skill should be used when the user asks to 'fix analysis', 'wrong results', 'notebook error', 'reviewer feedback', 'data changed', 'debug notebook', or needs mid-analysis course-correction for wrong results, notebook errors, or data changes."

Announce: "Using ds-fix for mid-analysis course correction."

Where This Fits

/ds (entry) → brainstorm → plan → implement → review → verify
                                    ↑
/ds-fix (midpoint) ─────────────────┘

This is the re-entry point. Jump back into a DS workflow that needs fixing.

<EXTREMELY-IMPORTANT> ## The Iron Law of DS Editing

DIAGNOSE BEFORE FIXING. This is not negotiable.

Before changing ANY analysis code, you MUST:

  1. Load the workflow context
  2. Identify WHAT is wrong (specific, not vague)
  3. Identify WHY it's wrong (root cause)
  4. Only THEN fix it (with output-first verification)

If you're about to change code without diagnosing first, STOP. </EXTREMELY-IMPORTANT>

Drive-Aligned Framing

Skipping diagnosis is NOT HELPFUL — guessing at a root cause wastes the user's time when the real bug resurfaces. Your guess is not a diagnosis. Your assumption is not evidence.

Drive-Aligned Framing (Consequences)

| Shortcut | Consequence | |----------|-------------| | Skipping diagnosis | You skipped diagnosis because you thought you were smart enough to guess. A checklist would outperform you. | | Fixing without understanding | You applied a patch without understanding the root cause. The bug returns in a different form — your fix was cosmetic. |

Rationalization Table

| Excuse | Reality | Do Instead | |--------|---------|------------| | "I can see the bug from the error message" | Error messages show symptoms, not causes. The traceback tells you WHERE, not WHY. | Trace backwards through the pipeline to find the first divergence point | | "This is the same bug as last time" | Similar symptoms can have different root causes. Pattern-matching from memory is not diagnosis. | Diagnose fresh — compare against LEARNINGS.md for prior context, but verify independently | | "The fix is obvious" | Obvious fixes mask deeper issues. The "obvious" fix is usually a patch over the symptom. | Identify root cause first, then fix. If it's truly obvious, diagnosis takes 30 seconds. | | "I'll just rerun and see if it works" | Retry without understanding is cargo cult debugging. If you don't know why it failed, you don't know why it passed. | Diagnose the failure, fix the cause, then rerun to verify | | "The data probably changed" | Verify before assuming external cause. Blaming data without evidence is a cop-out. | Compare data profiles before/after — shape, dtypes, distributions, nulls |

Red Flags - STOP Immediately If You Think:

| Thought | Why It's Wrong | Do Instead | |---------|----------------|------------| | "I'll just re-run everything" | Brute force hides root cause | Trace to the specific failing step | | "The data probably changed" | Your guess isn't a diagnosis | Compare data profiles before/after | | "Let me tweak the parameters" | Tweaking without understanding is p-hacking | Understand WHY parameters are wrong first | | "I'll fix it and check later" | You're skipping output-first | Fix AND verify in the same step |

Step 1: Load Context

Read workflow state, shared enforcement, AND shared check definitions:

Read("../../lib/references/ds-common-constraints.md")  # relative to this skill's base directory
Read(".claude/SPEC.md")
Read(".claude/PLAN.md")
Read(".claude/LEARNINGS.md")
Read("../../lib/skills/ds-implement/references/ds-checks.md")  # relative to this skill's base directory

The shared checks file contains data quality check definitions (DQ1-DQ6, M1, R1) used by both ds-review and ds-fix. Loading it here ensures the midpoint runs identical checks to the entry point's review phase. Without it, checks drift apart and the midpoint misses issues review would catch.

If no workflow state exists, suggest starting with /ds instead.

SAS Project Detection

After loading PLAN.md, check if Implementation Language is SAS or Mixed. If so, reload SAS enforcement before any fix:

Read("../wrds/references/sas-etl.md")

SAS projects have unique failure modes (hash merge memory, WHERE function wrapping, SGE array misconfiguration). The SAS enforcement must be loaded BEFORE diagnosing — otherwise you will misdiagnose SAS-specific issues as generic bugs.

Step 2: Diagnose

Identify the issue category:

| Category | Symptoms | Route To | |----------|----------|----------| | Runtime Error | Traceback, cell failure, import error | Debug Protocol | | Wrong Results | Numbers don't match expectations, sanity checks fail | Re-analysis Protocol | | Unclear Root Cause | 3+ plausible explanations, mysterious data quality issues | Competing Hypothesis Investigation | | Reviewer Feedback | Specific methodology concerns, requested changes | Revision Protocol | | Data Change | New data available, source updated, schema changed | Re-profiling Protocol | | Scope Change | New questions, expanded requirements | Spec Update → re-plan |

Ask user if ambiguous:

AskUserQuestion(questions=[
  {
    "question": "What needs fixing in your analysis?",
    "header": "Issue type",
    "options": [
      {"label": "Runtime error", "description": "Code fails, traceback, import error"},
      {"label": "Wrong results", "description": "Numbers don't look right, sanity checks fail"},
      {"label": "Unclear root cause", "description": "Multiple plausible explanations, mysterious data quality"},
      {"label": "Reviewer feedback", "description": "Specific changes requested by reviewer"},
      {"label": "Data/scope change", "description": "New data, updated requirements, new questions"}
    ],
    "multiSelect": false
  }
])

Diagnostic Routing Flowchart

┌───────────────────────┐
│  Load Context (Step 1) │
│  SPEC + PLAN + LEARN   │
└───────────┬───────────┘
            ▼
┌───────────────────────┐
│  Identify Symptoms     │
│  (Step 2)              │
└───────────┬───────────┘
            ▼
     ┌──────┴──────┐
     │  Traceback?  │─── YES ──→ Debug Protocol
     └──────┬──────┘              (Runtime Error)
            │ NO
            ▼
     ┌──────────────┐
     │ Numbers wrong │─── YES ──→ Re-analysis Protocol
     │ or unexpected?│              (Trace backwards)
     └──────┬───────┘
            │ NO
            ▼
     ┌──────────────┐
     │ 3+ plausible │─── YES ──→ Competing Hypothesis
     │ explanations?│              (Parallel investigation)
     └──────┬───────┘
            │ NO
            ▼
     ┌──────────────┐
     │ Reviewer      │─── YES ──→ Revision Protocol
     │ feedback?     │              (Fix per feedback)
     └──────┬───────┘
            │ NO
            ▼
     ┌──────────────┐
     │ Data or scope │─── YES ──→ Re-profiling / Spec Update
     │ changed?      │
     └──────────────┘

This flowchart IS the diagnostic spec. If the category table and flowchart disagree, the flowchart wins.

Step 3: Fix by Category

Runtime Error → Debug Protocol

  1. Identify the failing cell/script
  2. Read the error traceback
  3. Check LEARNINGS.md for prior context on this step
  4. Fix with output-first verification:
    • Print state BEFORE the failing operation
    • Fix the operation
    • Print state AFTER
    • Verify output matches expectations from PLAN.md

For notebook-specific errors, load notebook-debug patterns:

Read("../notebook-debug/SKILL.md")

Wrong Results → Re-analysis Protocol

<EXTREMELY-IMPORTANT> **Trace backwards. Do NOT guess forward.**

The bug is at the FIRST step where output diverges from expected. Find that step. </EXTREMELY-IMPORTANT>

  1. Identify WHICH results are wrong
  2. Trace backwards through the pipeline:
    • Check final output → is the aggregation wrong?
    • Check intermediate data → is the transformation wrong?
    • Check input data → is the loading/filtering wrong?
  3. Find the FIRST step where output diverges from expected
  4. Fix that step with output-first verification
  5. Re-run downstream steps and verify

Reviewer Feedback → Revision Protocol

  1. Document each piece of feedback as a task
  2. For each task:
    • Locate the relevant code/output
    • Apply the change
    • Verify with output-first protocol
    • Document in LEARNINGS.md
  3. After all changes, re-run review checks:
Read("../../lib/skills/ds-review/SKILL.md")

Data/Scope Change → Re-profiling Protocol

  1. Profile the new data (shape, dtypes, distributions)
  2. Compare to original data profile in LEARNINGS.md
  3. Identify what changed (schema, volume, distributions)
  4. Update PLAN.md if pipeline steps need changing
  5. Re-run affected pipeline steps with output-first verification
  6. Update LEARNINGS.md with new observations

Scope Change → Spec Update

If the question itself changed:

  1. Update SPEC.md with new objectives
  2. Review PLAN.md — which tasks are still valid?
  3. Add new tasks for new objectives
  4. Return to implement phase for new tasks

Unclear Root Cause → Competing Hypothesis Investigation

When 3+ plausible explanations exist, sequential investigation failed, or contradictory evidence is found, use the competing hypothesis protocol.

MANDATORY: Before proceeding, load the protocol:

Read("references/competing-hypothesis.md")

Key steps:

  1. Generate 3-5 competing hypotheses (Data Quality, Methodology, Implementation, Domain)
  2. Spawn investigation team (one agent per hypothesis)
  3. Scientific debate with cross-examination
  4. Evidence synthesis with confidence scoring (≥90 to proceed)
  5. Fix implementation via ds-delegate

Step 4: Verify Fix

After fixing, apply output-first verification:

  1. Re-run the fixed step: verify output shape, sample, stats
  2. Re-run downstream steps: verify no cascading issues
  3. Check against SPEC.md success criteria
  4. Update LEARNINGS.md with fix documentation

Gate: Fix Verification

Before claiming any fix is done, execute this gate:

1. IDENTIFY → What specific behavior was broken? (not vague — exact symptom)
2. RUN      → Execute the fixed code with output-first verification
3. READ     → Read the output: does it match SPEC.md success criteria?
4. VERIFY   → Re-run downstream steps — no cascading failures introduced
5. CLAIM    → Only declare "fix complete" if ALL gates pass

Skipping this gate means your fix is unverified. An unverified fix is not a fix — it's a guess.

Delegation

Main chat orchestrates. Task agents do the fixing:

Task(subagent_type="general-purpose", prompt="""
Fix [SPECIFIC ISSUE] in the analysis.

Context:
- Read .claude/LEARNINGS.md for prior steps
- Read .claude/PLAN.md for task details
- Read .claude/SPEC.md for objectives

Fix Protocol:
1. Trace to root cause (do NOT guess)
2. Fix with output-first verification
3. Print state BEFORE and AFTER fix
4. Re-run downstream steps
5. Update LEARNINGS.md

Report: what was wrong, what was fixed, verification output.
""")

When Fix Requires Rethinking

If the fix reveals the analysis approach is fundamentally wrong:

  1. Document findings in LEARNINGS.md
  2. Report to user: "The approach needs rethinking because [specific reason]"
  3. Suggest returning to /ds for re-planning

Don't try to salvage a broken approach with patches. A fresh plan costs less than cascading fixes.