Audit Echo Skill | Agent Skills

Audit Echo

Read ~/.claude/cartographer/audit.jsonl, filter to phase: "echo", and produce a findings report with severity tiers (CRITICAL/HIGH/MEDIUM/LOW) plus aggregate counts and tuning recommendations.

Dependencies

Path: ~/.claude/cartographer/audit.jsonl - the only event source.
Hook: ~/.claude/hooks/cartographer-echo.py writes those events. No hook, no events, so an empty report means "Echo is not running", not "Echo is quiet".
CLI: python3.

Step 1: Load events

python3 "${CLAUDE_SKILL_DIR}/scripts/report.py"

If the audit log is missing or empty: report LOW: no Echo events recorded in window and stop.

Step 2: Time-window aggregates

For each event, parse ts (ISO-8601 UTC). Compute counts grouped by decision (allow/deny/skip) and tool (Edit/Write/MultiEdit/Bash/mcp__serena__*) over the last 7 days and last 28 days.

Report shape:

| Window | Decision | Edit | Write | MultiEdit | Bash | mcp__serena__* | |--------|----------|-----:|------:|----------:|-----:|---------------:| | 7d | allow | | | | | | | 7d | deny | | | | | | | 7d | skip | | | | | | | 28d | ... | | | | | |

Step 3: Top-noisy symbols

For deny events, count (symbol, file) pairs that were later overridden by a second-attempt allow in the same session (matched on the hook's own retry signal: same file and symbol set, each retry consumed once). These are duplicate-detection-then-override cases: high candidates for stopword expansion. Note: pairing on the reason == "second-attempt" marker is exact; do not approximate it with an N-event proximity window (that inflates the override count).

Report the top 10 by deny-count, with second-attempt ratio. Anything with second-attempt ratio ≥50% over ≥5 occurrences is HIGH severity — propose adding to the stopword frozenset.

Step 4: False-positive sample

List (symbol, target_file, candidate_file:line) for every deny event with decision == "deny" followed by a second-attempt allow in the same session. Cap output at 20 examples (most recent first).

This is manual-review fodder; surface to user so they can tally TP vs FP and decide on tuning.

Step 5: Per-language breakdown

Infer language from the file extension (.py/.ts/.tsx/.js/.jsx/.rs/.go). Report deny/allow counts per language. Identify languages with disproportionate deny rates (>1.5× the overall mean) — MEDIUM: tree-sitter coverage may be over-broad for that grammar.

Step 6: Skip distribution

Count skip events by reason: settings, large-file, no-tree-sitter, test-file, unsupported-ext, mcp-unsupported, ripgrep-timeout, tree-sitter-parse-failed. Report counts.

HIGH severity if no-tree-sitter count > 0 (tree-sitter pack should be installed). MEDIUM if ripgrep-timeout > 1% of total events (the latency budget is at risk). LOW if mcp-unsupported > 0 (note the coverage gap for the next iteration).

Step 7: Findings report

Synthesize:

## Findings

### CRITICAL
(any item that breaks Echo's contract)

### HIGH
- {symbol X has 12 denies with 75% second-attempt rate — add to stopwords}
- {no-tree-sitter fired N times — fix package install}

### MEDIUM
- {Go shows 2.1× deny rate vs mean — investigate tree-sitter Go grammar coverage}
- {ripgrep-timeout > 1%}

### LOW
- {mcp-unsupported fires on `mcp__serena__create_text_file` — add tool_input shape handling}
- {N FP samples for manual review (see §4)}

## Summary

- Window: 7d
- Total events: ...
- Allow / deny / skip: ... / ... / ...
- Top noise symbol: ...
- Recommended next action: ...

Remediation snippets

For each HIGH finding, include a copy-paste-ready remediation block when possible. Example for stopword expansion:

# Edit ~/.claude/hooks/cartographer-echo.py
_STOPWORDS: frozenset[str] = frozenset({
    "__init__", "__main__", "main", "init", "setup", "run", "start", "stop",
    "new", "default", "clone", "eq", "hash", "to_string", "from_string",
    "<newword>",  # added by audit-echo on YYYY-MM-DD
})

Notes

Audit log path: ~/.claude/cartographer/audit.jsonl.
Event schema (PRD 00010 §Audit-log emission): ts, session, tool, file, decision, reason, symbols, matches, phase: "echo".
This skill is read-only; never mutate the audit log. Stopword/threshold tuning is a separate user action surfaced as remediation snippets.

Agent Skills: Audit Echo

Install this agent skill to your local

Skill Files