Agent-Skills.md

Agent Skills: Workflow: Regression Tests

Workflow for splitting a known compiler bug fix into auditable jj history. Use when the current or parent commit already contains a compiler fix and needs a preceding failing integration-test fixture plus a fix commit that updates the same checking, lowering, resolving, or LSP snapshot.

UncategorizedID: purefunctor/purescript-analyzer/workflow-regression-tests

Repository

purefunctor/purescript-analyzer
purefunctorLicense: NOASSERTION
9110

Install this agent skill to your local

pnpm dlx add-skill https://github.com/purefunctor/purescript-alexandrite/tree/HEAD/.agents/skills/workflow-regression-tests

Skill Files

Browse the full folder contents for workflow-regression-tests.

Download Skill

Loading file tree…

.agents/skills/workflow-regression-tests/SKILL.md

Skill Metadata

Name
workflow-regression-tests
Description
"Workflow for splitting a known compiler bug fix into auditable jj history. Use when the current or parent commit already contains a compiler fix and needs a preceding failing integration-test fixture plus a fix commit that updates the same checking, lowering, resolving, or LSP snapshot."

Workflow: Regression Tests

Use this when the fix is already known and the working copy, current commit, or parent commit contains the bug fix.

The goal is an auditable two-commit history:

  1. Failing fixture commit: add the fixture and accept a snapshot that captures the current undesirable behavior.
  2. Fix commit: keep the compiler fix and update the same snapshot so the undesirable behavior disappears or changes to the desired output.

This makes the regression visible in version control before the fix removes it.

Historical pattern

Historical examples in this repo follow this shape:

  • Add failing test case for bare row tail syntax → Fix inference for bare row tails in syntax
  • Add failing test case for constrained pattern scrutinee → Fix constrained pattern scrutinee checking
  • Add failing test case for open row matching → Fix open row tail instance matching

The fix commit usually touches compiler code plus the same .snap. Depending on the integration-test category, the snapshot diff might remove a Diagnostics block, replace an incorrect inferred result, update lowered/resolved bindings, or change an LSP hover/completion/definition report.

Choose the integration-test category

Use the category that owns the observable regression:

| Category | Alias | Use when the regression is visible in | |----------|-------|----------------------------------------| | checking | c | Type checking, inference, kinding, roles, constraints, or checking diagnostics | | lowering | l | Lowered core output, equation/binder shape, or source-to-core links | | resolving | r | Name resolution, imports, exports, qualification, re-exports, or resolver diagnostics | | lsp | - | Hover, definition, completion, import edits, or editor-facing source positions |

Use the workflow-integration-tests skill for fixture authoring details. Use the command reference at .agents/skills/workflow-integration-tests/reference/compiler-scripts.md for runner syntax, filters, snapshots, and trace debugging.

Workflow

1. Preserve the existing fix commit

If the current commit contains the intended fix, give it a clear description first:

jj describe -m "Fix <bug>"

Then insert a parent commit before it for the failing fixture:

jj new -B @ -m "Add failing test case for <bug>"

Use the existing commit message style when a more specific noun reads better, such as Add <bug> regression fixture.

2. Add the regression fixture

Create a fixture in the selected category with a descriptive name:

just t <category> --create "<descriptive name>"

Write a focused PureScript fixture that reproduces one behavior. Use Main.purs by default; add supporting modules only when imports, exports, qualification, or LSP candidates are part of the regression.

Snapshot expectations differ by category:

  • checking and lsp snapshot Main.purs only.
  • lowering and resolving snapshot every .purs file in the fixture.

Accept the snapshot in the failing fixture commit:

just t <category> NNN --accept

The snapshot should intentionally encode the bug: the wrong error, missing type, bad constraint, incorrect lowered output, wrong resolution target, incorrect LSP response, unexpected ???, or other undesirable current behavior. Do not try to make this commit green by changing compiler behavior; its purpose is to record the failure before the fix.

3. Return to the fix commit and update the snapshot

Move back to the child commit containing the fix:

jj edit <fix-change-id>

Run the same fixture and inspect the snapshot change:

just t <category> NNN --diff

Accept the updated snapshot:

just t <category> NNN --accept

The diff should show the undesirable behavior being removed or replaced by the correct behavior. This is commonly a deleted Diagnostics section or a targeted type/constraint change in the same snapshot added by the previous commit.

Snapshot review checklist

Before finishing, verify:

  • The failing fixture commit snapshot captures the bug clearly.
  • The fix commit snapshot changes only what the fix should change.
  • Unexpected ??? does not appear unless it is the regression being documented.
  • Error kinds, locations, inferred types, constraints, lowered bindings, resolved references, and LSP payloads are intentional for the selected category.
  • The fixture is narrow enough for the snapshot diff to be easy to audit.
  • For lowering and resolving, every changed module snapshot in the fixture is expected.

Useful commands

just t <category> NNN           # Run fixture
just t <category> NNN --diff    # Inspect snapshot diff
just t <category> NNN --accept  # Accept fixture snapshot
just t <category> NNN --reject  # Reject fixture snapshot