Agent-Skills.md

Agent Skills: Evaluate Fusion MCP Search Index

Iterative evaluation of Fusion Framework MCP search quality against documented domain patterns. Loads domain files from eval/index/, queries Fusion MCP for each pattern, validates recall against must/should requirements, and produces a human-readable pass/fail report. USE FOR: eval core, eval all, evaluate MCP index accuracy, validate search recall for a domain, check index freshness. DO NOT USE FOR: writing domain patterns, populating eval/index/ files, running CI pipelines, or batch automation.

UncategorizedID: equinor/fusion-framework/custom-index-eval

Repository

equinor/fusion-framework
equinorLicense: MIT
98

Install this agent skill to your local

pnpm dlx add-skill https://github.com/equinor/fusion-framework/tree/HEAD/.agents/skills/custom-index-eval

Skill Files

Browse the full folder contents for custom-index-eval.

Download Skill

Loading file tree…

.agents/skills/custom-index-eval/SKILL.md

Skill Metadata

Name
custom-index-eval
Description
'Evaluates Fusion MCP search quality against eval/index domain files. USE FOR: eval core, eval all, validate MCP index recall, check search freshness, verify documented framework patterns. DO NOT USE FOR: authoring eval patterns, editing docs, CI automation, or application-development search answers.'

Evaluate Fusion MCP Search Index

When To Use

Use when the user wants a pass/fail read on whether Fusion MCP returns the expected framework knowledge for documented query patterns.

Trigger phrases:

  • eval core
  • eval <domain>
  • eval all
  • "check MCP index accuracy"
  • "validate search recall"
  • "is the index stale?"

Inputs

  • Domain target: all or a file stem under eval/index/, such as core.
  • Strictness: full by default, or strict for must expectations only.

Ask one question if the user says only eval.

Workflow

  1. Resolve domain files:

    • <domain> maps to eval/index/<domain>.md.
    • all means every markdown file in eval/index/ except README.md.
    • Skip empty files or files without ## query headings.

  2. Parse each domain file:

    • # heading is the domain name.
    • Text under # is judgement guidance for every query.
    • Each ## heading is the MCP query.
    • - must ... and - should ... bullets are expectations.

  3. Judge each query:

    • Prefer the agents/query-judge.md sub-agent.
    • If sub-agents are unavailable, run the same flow inline.
    • Search with mcp_fusion_search_framework first, then mcp_fusion_search if needed.
    • Verdicts are pass, partial, or fail; never inflate ambiguous results.

  4. Report using assets/report-template.md:

    • Include date, domain file, strictness, verdict table, summary counts, and recommendations.
    • For all, lead with per-domain summary before detailed results.
    • Call out must failures as critical index gaps.

Safety

  • Read-only repository behavior; do not edit eval files, docs, or index content.
  • MCP calls are read-only.
  • Stop clearly if MCP is unavailable or rate-limited.
  • Do not cache verdicts across sessions; results represent current index state.

Expected Output

  • Per-query verdicts with one-sentence explanations
  • Must/should satisfaction counts
  • Pass rate and failure summary
  • Concrete recommendations for stale, missing, or weak search coverage