Known Build Error History Analysis Skill

Known Build Error History Analysis

Reconstruct failure timelines for "Known Build Error" issues by analyzing the edit history of the issue body's hit count table. The build-analysis bot periodically edits these issues to update rolling hit counts (24h/7d/1mo) — this skill mines those edits to recover the full failure history.

When to Use This Skill

Use this skill when:

Investigating whether a known issue is still actively failing ("when did this last fail?")
Checking failure frequency and trends for a flaky test
Triaging known issues — identifying which are dormant vs actively hitting
Deciding whether to close a known issue that appears inactive
Scanning all open known issues to find the most problematic ones

Workflow Steps

Single Issue Analysis

Determine the issue number and repository from the user's request
Run the script: ./scripts/Get-KnownIssueHistory.ps1 -IssueNumber <N> -Repository <owner/repo>
Add -Since <date> if the user only wants recent failures, or -Json for structured output
Summarize the results: total failure events, failure periods, last failure date, and days since last failure
If posting results to an issue or PR, include a link to this skill (see Posting Results below)

Scanning All Active Known Issues

Run the script: ./scripts/Get-KnownIssueHistory.ps1 -ListActive -Repository <owner/repo>
Add -MaxIssues <N> to limit scope (default 50, max 100)
Summarize the ranked table highlighting the most actively failing issues and any dormant candidates for closing

Quick Start

# Analyze a single known issue
./scripts/Get-KnownIssueHistory.ps1 -IssueNumber 100088

# Only show recent failures
./scripts/Get-KnownIssueHistory.ps1 -IssueNumber 100088 -Since 2025-01-01

# JSON output for programmatic use
./scripts/Get-KnownIssueHistory.ps1 -IssueNumber 100088 -Json

# Scan all open Known Build Error issues, ranked by activity
./scripts/Get-KnownIssueHistory.ps1 -ListActive

# Different repository
./scripts/Get-KnownIssueHistory.ps1 -IssueNumber 12345 -Repository dotnet/aspnetcore

Key Parameters

| Parameter | Description | |-----------|-------------| | -IssueNumber | Known Build Error issue number to analyze | | -Repository | Target repo in owner/repo format (default: dotnet/runtime) | | -ListActive | Scan all open Known Build Error issues and rank by recent activity | | -MaxIssues | Max issues to scan in ListActive mode (default: 50) | | -Since | Only show failures after this date (ISO 8601, e.g. 2025-01-01) | | -Json | Output structured JSON ([KNOWN_ISSUE_HISTORY] block) instead of table |

Modes

Single Issue Mode (`-IssueNumber`)

Queries the GitHub GraphQL userContentEdits API for the issue, parses each edit's hit count table, and detects failure events by tracking when the 24-hour count transitions from 0 to a positive value.

Output includes:

Failure summary — total failure events, failure periods, last failure date, days since last failure
Failure periods — clusters of failures grouped by the gap between all-clear events, with peak hit counts
Edit timeline — chronological table of every hit count snapshot with failure event markers

List Active Mode (`-ListActive`)

Scans all open issues with the "Known Build Error" label in the repository and analyzes each one's edit history. Outputs a ranked table sorted by most recent failure.

Use this to find:

Which known issues are still actively failing
Which known issues haven't failed in months (candidates for closing)
Overall flaky test health of the repository

Interpreting Results

Failure Events

| Marker | Meaning | |--------|---------| | <-- NEW FAILURE | 24h count went from 0 to N — a build just matched this known issue | | <-- ADDITIONAL FAILURE | 24h count increased — another build matched within 24 hours | | (cleared) | All counts returned to 0 — no recent matches |

Failure Periods

Consecutive failure events are grouped into periods. A period starts when 24h goes from 0→N and ends when all counts return to 0.

Many short periods → intermittently flaky test
One long period → sustained regression
Peak 24h count → severity indicator (1 = occasional, higher = frequent across builds)

Triage Guidelines

| Days Since Last Failure | Status | Action | |------------------------|--------|--------| | 0-7 | Actively failing | Needs investigation | | 7-30 | Recently active | Monitor | | 30-90 | Dormant | Consider closing | | 90+ | Likely fixed | Recommend closing |

Posting Results

When posting results to an issue or PR comment, always include a link back to this skill so readers know where the analysis came from:

Analysis generated by [known-issue-history](https://github.com/dotnet/arcade-skills/tree/main/plugins/dotnet-dnceng/skills/known-issue-history)

How It Works

The build-analysis bot edits Known Build Error issue bodies whenever hit counts change. GitHub stores the revision history via the GraphQL userContentEdits API. Each edit includes a diff showing what changed in the issue body. The script parses hit-count table rows from these diffs to reconstruct snapshots:

|24-Hour Hit Count|7-Day Hit Count|1-Month Count|
|---|---|---|
|0|0|0|

The script extracts these snapshots chronologically and detects state transitions to reconstruct when failures actually occurred — recovering historical data that the rolling windows would otherwise hide.

Limitations

Edit retention: GitHub may not retain edits indefinitely for very old issues
Polling frequency: Edits occur when hit counts change, not on a fixed schedule — timestamps show when the bot detected changes, not the exact test failure time
Rolling windows: A single failure appears across multiple snapshots as it moves through the 24h→7d→1mo windows

References

Detailed approach and interpretation: See references/known-issue-history.md
Known Issues documentation: See dotnet/arcade Known Issues guide
Build Analysis website: https://helix.dot.net/BuildAnalysis
Known Issues board: https://github.com/orgs/dotnet/projects/111

Agent Skills: Known Build Error History Analysis

Install this agent skill to your local

Skill Files