AI Health Check Skill | Agent Skills

AI Health Check

Before you ship an AI feature, it needs to pass 6 checks.

Most AI products fail because PMs skip the basics: no cost model, broken failure UX, terrible data quality. This skill stops you from launching garbage.

Entry Point

When this skill is invoked, start with:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 AI HEALTH CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Before shipping an AI feature, it needs to pass 6 checks.

What AI feature are you preparing to launch?

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Usage

/ai-health-check [feature-name]

Examples:

/ai-health-check "AI product recommendations" - Audit specific feature
/ai-health-check "email composer AI" - Manual description
/ai-health-check --pre-launch - Full checklist for current sprint

What Happens

Invokes the ai-implementation-auditor agent
Asks hard questions about your AI feature
Grades each of 6 dimensions: Ready / Risk / Blocker
Tells you if you can ship

The 6 Dimensions

| Dimension | What It Checks | |-----------|---------------| | Model Selection | Did you try simple approaches first? | | Data Quality | The thing you're probably ignoring | | Cost Modeling | Can you afford this at scale? | | Production Monitoring | How will you know if it breaks? | | Failure UX | What happens when AI screws up? | | System Optimization | Are you measuring the right things? |

Verdict Logic

| Condition | Verdict | |-----------|---------| | Any Blocker | DON'T SHIP | | 2+ Risks (no blockers) | NEEDS WORK | | 0-1 Risks | READY |

Sample Output

AI Health Check: Email Composer

Overall Readiness: NEEDS WORK (4/6 dimensions ready)

---

Ready: Model Selection, Production Monitoring, System Optimization
Risk: Data Quality, Failure UX
Blocker: Cost Modeling

VERDICT: DON'T SHIP YET

You have 1 blocker:
- No cost model -> Run /ai-cost-check RIGHT NOW

You have 2 risks:
- Data quality strategy undefined
- Failure UX is broken ("Something went wrong" isn't helpful)

---

What To Do Now:

Option A: Fix everything (RECOMMENDED)
1. Run /ai-cost-check (10 min)
2. Define data quality strategy (2 hours)
3. Build better failure UX (3 hours)
4. Rerun /ai-health-check

Option B: Ship with known risks
1. Fix the blocker only
2. Ship knowing data quality and failure UX are weak
3. Plan to fix in week 1

Common Blockers

Cost Modeling missing:

"You're about to launch with zero idea if this bankrupts you at scale." Run /ai-cost-check first.

Failure UX broken:

"Something went wrong" tells users nothing. No confidence indicators = users don't know when to trust the AI.

No monitoring plan:

"Launching without monitoring = flying blind."

Philosophy (Chip Huyen)

"Most AI failures are UX problems, not technical ones."
"Data quality beats tool selection."
"Fine-tuning should be your last resort."
"The gap between a demo and a product is production engineering."

Related Commands

/ai-cost-check - Detailed cost modeling (run if cost dimension is blocked)
/start-evals - Set up quality testing
/four-risks - Overall feature risk assessment

Best for: Pre-launch validation of AI features Key insight: "Fine-tuning is the last resort. Data quality beats tool selection. Most AI failures are UX problems."

Agent Skills: AI Health Check

Install this agent skill to your local

Skill Files