Convergence Diagnostics Skill

Convergence Diagnostics

Use this skill when checking MCMC convergence after fitting Stan models. Convergence means chains mixed and explored the same target, and you have enough effective draws.

CmdStanPy Diagnostics

After fitting with CmdStanPy, run:

fit.summary(): Returns DataFrame with R-hat, ESS_bulk, ESS_tail, MCSE per parameter
fit.diagnose(): Checks for divergences, max treedepth, low E-BFMI, low ESS, high R-hat

If diagnose() reports no problems, you still need visual checks via ArviZ.

ArviZ Workflow

Convert to InferenceData:

idata = az.from_cmdstanpy(
    fit,
    log_likelihood="log_lik",
    observed_data={"y": y_obs}
)

Run numerical diagnostics:

az.rhat(idata): Rank-normalized split R-hat
az.ess(idata): Bulk and tail effective sample size
az.bfmi(idata): Bayesian fraction of missing information
az.mcse(idata): Monte Carlo standard error
az.summary(idata): All diagnostics in one table

Thresholds

Must achieve:

R̂ < 1.01 (all parameters) - measures chain agreement
ESS bulk and tail ≥ 400 per parameter - enough effective draws
BFMI ≥ 0.3 per chain - adequate energy exploration
MCSE << posterior SD - Monte Carlo error small relative to uncertainty
No divergent transitions after warmup

Visual Diagnostics

Chain mixing and stationarity:

az.plot_trace(): Should show "fat fuzzy caterpillars", no trends or stuck chains. Divergences shown as vertical lines.
az.plot_rank(): Rank histograms should be uniform and similar across chains. U-shapes or skew indicate poor mixing.

Autocorrelation and ESS:

az.plot_autocorr(): Slow decay indicates high correlation and low ESS
az.plot_ess(kind="evolution"): ESS growth over draws - should keep climbing
az.plot_ess(kind="local"): ESS in local windows/quantiles - checks tail exploration

HMC-specific pathologies:

az.plot_energy(): Overlays energy transitions vs marginal energy. Low BFMI shows mismatch.
az.plot_pair(divergences=True): Localizes divergences in parameter space (funnels, tight correlations)
az.plot_parallel(): Parallel coordinates showing divergent vs non-divergent draws

Common Issues

Divergences + low BFMI: Geometry problems (funnels, stiff regions). Reparameterize or increase adapt_delta.
High R̂, good visuals: Chains haven't run long enough. Extend iterations.
Low ESS, good R̂: High autocorrelation. Reparameterize or run longer.
Max treedepth warnings: Strong correlations. Reparameterize or simplify model.
Multimodality in plot_posterior: Identification problem or multiple modes.

Remember: You never prove convergence, only build a strong circumstantial case. The sampler tells you about your model - listen to it.

Agent Skills: Convergence Diagnostics

Install this agent skill to your local

Skill Files