QA Testing Strategy (Jan 2026) Skill

QA Testing Strategy (Jan 2026)

Risk-based quality engineering strategy for modern software delivery.

Core references: curated links in data/sources.json (SLOs/error budgets, contracts, E2E, OpenTelemetry). Start with references/operational-playbook.md for a compact, navigable overview.

Scope

Create or update a risk-based test strategy (what to test, where, and why)
Define quality gates and release criteria (merge vs deploy)
Select the smallest effective layer (unit → integration → contract → E2E)
Make failures diagnosable (artifacts, logs/traces, ownership)
Operationalize reliability (flake SLO, quarantines, suite budgets)

Use Instead

| Need | Skill | |------|-------| | Debug failing tests or incidents | qa-debugging | | Test LLM agents/personas | qa-agent-testing | | Perform security audit/threat model | software-security-appsec | | Design CI/CD pipelines and infra | ops-devops-platform |

Quick Reference

| Test Type | Goal | Typical Use | |-----------|------|-------------| | Unit | Prove logic and invariants fast | Pure functions, core business rules | | Component | Validate UI behavior in isolation | UI components and state transitions | | Integration | Validate boundaries with real deps | API + DB, queues, external adapters | | Contract | Prevent breaking changes cross-team | OpenAPI/AsyncAPI/JSON Schema/Protobuf | | E2E | Validate critical user journeys | 1–2 “money paths” per product area | | Performance | Enforce budgets and capacity | Load, stress, soak, regression trends | | Visual | Catch UI regressions | Layout/visual diffs on stable pages | | Accessibility | Automate WCAG checks | axe smoke + targeted manual audits | | Security | Catch common web vulns early | DAST smoke + critical checks in CI |

Default Workflow

Clarify scope and risk: critical journeys, failure modes, and non-functional risks (latency, data loss, auth).
Define quality signals: SLOs/error budgets, contract/schema checks, and what blocks merge vs blocks deploy.
Choose the smallest effective layer (unit → integration → contract → E2E).
Make failures diagnosable: artifacts + correlation IDs (logs/traces/screenshots), clear ownership, deflake runbook.
Operationalize: flake SLO, quarantine with expiry, suite budgets (PR gate vs scheduled), dashboards.

Test Pyramid

           /\
          /E2E\          5-10% - Critical journeys
         /------\
        /Integr. \       15-25% - API, DB, queues
       /----------\
      /Component \       20-30% - UI modules
     /------------\
    /   Unit      \      40-60% - Logic and invariants
   /--------------\

Decision Tree: Test Strategy

Need to test: [Feature Type]
    │
    ├─ Pure business logic/invariants? → Unit tests (mock boundaries)
    │
    ├─ UI component/state transitions? → Component tests
    │   └─ Cross-page user journey? → E2E tests
    │
    ├─ API Endpoint?
    │   ├─ Single service boundary? → Integration tests (real DB/deps)
    │   └─ Cross-service compatibility? → Contract tests (schema/versioning)
    │
    ├─ Event-driven/API schema evolution? → Contract + backward-compat tests
    │
    └─ Performance-critical? → k6 load testing

Core QA Principles

Definition of Done

Strategy is risk-based: critical journeys + failure modes explicit
Test portfolio is layered: fast checks catch most defects
CI is economical: fast pre-merge gates, heavy suites scheduled
Failures are diagnosable: actionable artifacts (logs/trace/screenshots)
Flakes managed with SLO and deflake runbook

Shift-Left Gates (Pre-Merge)

Contracts: OpenAPI/AsyncAPI/JSON Schema validation
Static checks: lint, typecheck, secret scanning
Fast tests: unit + key integration (avoid full E2E as PR gate)

Shift-Right (Post-Deploy)

Synthetic checks for critical paths (monitoring-as-tests)
Canary analysis: compare SLO signals and key metrics before ramping
Feature flags for safe rollouts and fast rollback
Convert incidents into regression tests (prefer lower layers first)

CI Economics

| Budget | Target | |--------|--------| | PR gate | p50 ≤ 10 min, p95 ≤ 20 min | | Mainline health | ≥ 99% green builds/day |

Flake Management

Define: test fails without product change, passes on rerun
Track weekly: flaky_failures / total_test_executions (where flaky_failure = fail_then_pass_on_rerun)
SLO: Suite flake rate ≤ 1% weekly
Quarantine policy with owner and expiry
Use the deflake runbook: template-flaky-test-triage-deflake-runbook.md

Common Patterns

AAA Pattern

it('should apply discount', () => {
  // Arrange
  const order = { total: 150 };
  // Act
  const result = calculateDiscount(order);
  // Assert
  expect(result.discount).toBe(15);
});

Page Object Model (E2E)

class LoginPage {
  async login(email: string, password: string) {
    await this.page.fill('[data-testid="email"]', email);
    await this.page.fill('[data-testid="password"]', password);
    await this.page.click('[data-testid="submit"]');
  }
}

Anti-Patterns

| Anti-Pattern | Problem | Solution | |--------------|---------|----------| | Testing implementation | Breaks on refactor | Test behavior | | Shared mutable state | Flaky tests | Isolate test data | | sleep() in tests | Slow, unreliable | Use proper waits | | Everything E2E | Slow, expensive | Use test pyramid | | Ignoring flaky tests | False confidence | Fix or quarantine |

Do / Avoid

Do

Write tests against stable contracts and user-visible behavior
Treat flaky tests as P1 reliability work
Make "how to debug this failure" part of every suite

Avoid

"Everything E2E" as default
Sleeps/time-based waits (use event-based)
Coverage % as primary quality KPI

Feature Matrix vs Test Matrix Gate (Release Blocking)

Before release, run a coverage audit that maps product features/backlog IDs to direct test evidence.

Gate Rules

Every release-scoped feature must map to at least one direct automated test, or an explicit waiver with owner/date.
Evidence must include file path and test identifier (suite/spec/case).
"Covered indirectly" is not accepted without written rationale and risk acknowledgment.
If critical features have no direct evidence, release is blocked.

Minimal Audit Output

feature/backlog id
coverage status (direct, indirect, none)
evidence reference
risk level
owner and due date for gaps

Resources

| Resource | Purpose | |----------|---------| | comprehensive-testing-guide.md | End-to-end playbook across layers | | operational-playbook.md | Testing pyramid, BDD, CI gates | | shift-left-testing.md | Contract-first, BDD, continuous testing | | test-automation-patterns.md | Reliable patterns and anti-patterns | | playwright-webapp-testing.md | Playwright patterns | | chaos-resilience-testing.md | Chaos engineering | | observability-driven-testing.md | OpenTelemetry, trace-based | | contract-testing-2026.md | Pact, Specmatic | | synthetic-test-data.md | Privacy-safe, ephemeral test data | | test-environment-management.md | Environment provisioning and lifecycle | | quality-metrics-dashboard.md | Quality metrics and dashboards | | compliance-testing.md | SOC2, HIPAA, GDPR, PCI-DSS testing | | feature-matrix-vs-test-matrix-gate.md | Release-blocking feature-to-test coverage audit |

Templates

| Template | Purpose | |----------|---------| | template-test-case-design.md | Given/When/Then and test oracles | | test-strategy-template.md | Risk-based strategy | | template-flaky-test-triage.md | Flake triage runbook | | template-jest-vitest.md | Unit test patterns | | template-api-integration.md | API + DB integration tests | | template-playwright.md | Playwright E2E | | template-visual-testing.md | Visual regression testing | | template-k6-load-testing.md | k6 performance | | automation-pipeline-template.md | CI stages, budgets, gates | | template-cucumber-gherkin.md | BDD feature files and steps | | template-release-coverage-audit.md | Feature matrix vs test matrix release audit |

Data

| File | Purpose | |------|---------| | sources.json | External references |

Related Skills

qa-debugging — Debugging failing tests
qa-agent-testing — Testing AI agents
software-backend — API patterns to test
ops-devops-platform — CI/CD pipelines

Ops Gate: Release-Safe Verification Sequence

Use this sequence for feature branches that touch user flows, pricing, localization, or analytics.

# 1) Static checks
npm run lint
npm run typecheck

# 2) Fast correctness
npm run test:unit

# 3) Critical path checks
npm run test:e2e -- --grep "@critical"

# 4) Instrumentation gate (if configured)
npm run test:analytics-gate

# 5) Production build
npm run build

If a Gate Fails

Capture exact failing command and first error line.
Classify: environment issue, baseline known failure, or regression.
Re-run only the failed gate once after fix.
Do not continue to later gates while earlier required gates are red.

Agent Output Contract for QA Handoff

Always report:

commands run,
pass/fail per gate,
whether failures are pre-existing or introduced,
next blocking action.

Fact-Checking

Use web search/web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.
Prefer primary sources; report source links and dates for volatile information.
If web access is unavailable, state the limitation and mark guidance as unverified.

Agent Skills: QA Testing Strategy (Jan 2026)

Install this agent skill to your local

Skill Files