Fishbone · Diagram Design

Checkout p99 latency · Root-cause fishbone

Five candidate cause categories investigated after a 90-minute checkout latency incident. The confirmed root cause carries the diagram's only accent — everything else stays in the neutral ink/muted palette so the reader's eye lands on the answer, not the investigation.

Checkout p99 latency · Root-cause fishbone Fishbone diagram tracing a checkout latency incident to five candidate cause categories, with a missing database index under Data confirmed as the root cause. canary skipped config drift index missing on orders.created_at table 3x forecast connection pool cap 20 GC pauses no p99 alert dashboard sampled at 5m on-call handover mid-incident DEPLOY DATA RUNTIME OBSERVABILITY PEOPLE EFFECT Checkout p99 latency 4x for 90 minutes LEGEND Category (bone) Confirmed root cause Sub-cause Observed effect

ROOT CAUSE

Data — missing index

No index on orders.created_at; the query planner fell back to a sequential scan once the table crossed 3x its forecast size. The single confirmed cause — every other category was ruled out.

Contributing categories

Deploy (canary skipped, config drift), Runtime (pool cap 20, GC pauses), and Observability (no p99 alert, 5-minute dashboard sampling) all slowed detection but didn't cause the regression themselves.

People

A mid-incident on-call handover cost roughly ten minutes of context — worth fixing, not the root cause.

checkout p99 latency · root-cause fishbone example · diagram design