Mutation Testing Skill | Agent Skills

Mutation Testing

Mutation testing answers: "Would my tests catch this bug?" by actually introducing bugs and running tests.

Execution Workflow

CRITICAL: This skill actually mutates code and runs tests. Follow this exact process:

Step 1: Identify Target Code

# Get changed files on the branch
git diff main...HEAD --name-only | grep -E '\.(ts|js|tsx|jsx|vue)$' | grep -v '\.test\.' | grep -v '\.spec\.'

Step 2: For Each Function to Test

Execute this loop for each mutation:

1. READ the original file and note exact content
2. APPLY one mutation (edit the code)
3. RUN tests: pnpm test --run (or specific test file)
4. RECORD result: KILLED (test failed) or SURVIVED (test passed)
5. RESTORE original code immediately
6. Repeat for next mutation

Step 3: Report Results

After all mutations, provide a summary table:

| Mutation | Location | Result | Action Needed |
|----------|----------|--------|---------------|
| `>` → `>=` | file.ts:42 | SURVIVED | Add boundary test |
| `&&` → `||` | file.ts:58 | KILLED | None |

Mutation Operators to Apply

Priority 1: Boundary Mutations (Most Likely to Survive)

| Original | Mutate To | Why It Matters | |----------|-----------|----------------| | < | <= | Boundary not tested | | > | >= | Boundary not tested | | <= | < | Equality case missed | | >= | > | Equality case missed |

Priority 2: Boolean Logic Mutations

| Original | Mutate To | Why It Matters | |----------|-----------|----------------| | && | \|\| | Only tested when both true | | \|\| | && | Only tested when both false | | !condition | condition | Negation not verified |

Priority 3: Arithmetic Mutations

| Original | Mutate To | Why It Matters | |----------|-----------|----------------| | + | - | Tested with 0 only | | - | + | Tested with 0 only | | * | / | Tested with 1 only |

Priority 4: Return/Early Exit Mutations

| Original | Mutate To | Why It Matters | |----------|-----------|----------------| | return x | return null | Return value not asserted | | return true | return false | Boolean return not checked | | if (cond) return | // removed | Early exit not tested |

Priority 5: Statement Removal

| Original | Mutate To | Why It Matters | |----------|-----------|----------------| | array.push(x) | // removed | Side effect not verified | | await save(x) | // removed | Async operation not verified | | emit('event') | // removed | Event emission not tested |

Practical Execution Example

Example: Testing a Validation Function

Original code (src/utils/validation.ts:15):

export function isValidAge(age: number): boolean {
  return age >= 18 && age <= 120;
}

Mutation 1: Change >= to >

export function isValidAge(age: number): boolean {
  return age > 18 && age <= 120;  // MUTATED
}

Run tests:

pnpm test --run src/__tests__/validation.test.ts

Result: Tests PASS → SURVIVED (Bad! Need test for isValidAge(18))

Restore original code immediately

Mutation 2: Change && to ||

export function isValidAge(age: number): boolean {
  return age >= 18 || age <= 120;  // MUTATED
}

Run tests:

pnpm test --run src/__tests__/validation.test.ts

Result: Tests FAIL → KILLED (Good! Tests catch this bug)

Restore original code immediately

Results Interpretation

Mutant States

| State | Meaning | Action | |-------|---------|--------| | KILLED | Test failed with mutant | Tests are effective | | SURVIVED | Tests passed with mutant | Add or strengthen test | | TIMEOUT | Tests hung (infinite loop) | Counts as detected |

Mutation Score

Score = (Killed + Timeout) / Total Mutations * 100

| Score | Quality | |-------|---------| | < 60% | Weak - significant test gaps | | 60-80% | Moderate - improvements needed | | 80-90% | Good - minor gaps | | > 90% | Strong test suite |

Fixing Surviving Mutants

When a mutant survives, add a test that would catch it:

Surviving: Boundary mutation (`>=` → `>`)

// Add boundary test
it('accepts exactly 18 years old', () => {
  expect(isValidAge(18)).toBe(true);  // Would fail if >= became >
});

Surviving: Logic mutation (`&&` → `||`)

// Add test with mixed conditions
it('rejects when only one condition met', () => {
  expect(isValidAge(15)).toBe(false);  // Would pass if && became ||
});

Surviving: Statement removal

// Add side effect verification
it('saves to database', async () => {
  await processOrder(order);
  expect(db.save).toHaveBeenCalledWith(order);  // Would fail if save removed
});

Quick Checklist During Mutation

For each mutation, ask:

Before mutating: Does a test exist for this code path?
After running tests: Did any test actually fail?
If survived: What specific test would catch this?
After fixing: Re-run mutation to confirm killed

Common Surviving Mutation Patterns

Tests Only Check Happy Path

// WEAK: Only tests success case
it('validates', () => {
  expect(validate(goodInput)).toBe(true);
});

// STRONG: Tests both cases
it('validates good input', () => {
  expect(validate(goodInput)).toBe(true);
});
it('rejects bad input', () => {
  expect(validate(badInput)).toBe(false);
});

Tests Use Identity Values

// WEAK: Mutation survives
expect(multiply(5, 1)).toBe(5);  // 5*1 = 5/1 = 5

// STRONG: Mutation detected
expect(multiply(5, 3)).toBe(15);  // 5*3 ≠ 5/3

Tests Don't Assert Return Values

// WEAK: No return value check
it('processes', () => {
  process(data);  // No assertion!
});

// STRONG: Asserts outcome
it('processes', () => {
  const result = process(data);
  expect(result).toEqual(expected);
});

Important Rules

ALWAYS restore original code after each mutation
Run tests immediately after applying mutation
One mutation at a time - don't combine mutations
Focus on changed code - prioritize branch diff
Track all results - report full mutation summary

Summary Report Template

After completing mutation testing, provide:

## Mutation Testing Results

**Target**: `src/features/workout/utils.ts` (functions: X, Y, Z)
**Total Mutations**: 12
**Killed**: 9
**Survived**: 3
**Score**: 75%

### Surviving Mutants (Action Required)

| # | Location | Original | Mutated | Suggested Test |
|---|----------|----------|---------|----------------|
| 1 | line 42 | `>=` | `>` | Test boundary value |
| 2 | line 58 | `&&` | `\|\|` | Test mixed conditions |
| 3 | line 71 | `emit()` | removed | Verify event emission |

### Killed Mutants (Tests Effective)

- Line 35: `+` → `-` killed by `calculation.test.ts`
- Line 48: `true` → `false` killed by `validate.test.ts`
- ...

Related Skills

systematic-debugging - Root cause analysis
testing-conventions - Query priority, expect.poll()
vue-integration-testing - Page objects, browser mode
vitest-mocking - Test doubles and mocking patterns

Agent Skills: Mutation Testing

Install this agent skill to your local

Skill Files