Agent-Skills.md

Agent Skills: Searching Codebases

>-

UncategorizedID: oaustegard/claude-skills/searching-codebases

Repository

oaustegard/claude-skills
oaustegardLicense: MIT
1287

Install this agent skill to your local

pnpm dlx add-skill https://github.com/oaustegard/claude-skills/tree/HEAD/searching-codebases

Skill Files

Browse the full folder contents for searching-codebases.

Download Skill

Loading file tree…

searching-codebases/SKILL.md

Skill Metadata

Name
searching-codebases
Description
>-

Searching Codebases

Find code in any codebase by pattern or concept. One entry point, two search strategies, automatic routing.

Prerequisites

uv tool install ripgrep

tree-sitting installs automatically when needed — for --expand context expansion and for the binding-resolved --refs/--def/--hover tier, which uses it to resolve symbol positions. Only the bare tree-sitter package is fetched; the language grammars ship bundled.

Primary Command

SKILL_DIR=/mnt/skills/user/searching-codebases

python3 $SKILL_DIR/scripts/search.py SOURCE "query1" ["query2" ...] [OPTIONS]

SOURCE is any of:

  • Local directory path
  • GitHub URL (downloads tarball automatically)
  • uploads (uses /mnt/user-data/uploads/)
  • project (uses /mnt/project/)
  • Path to a .zip or .tar.gz archive

Search Modes

Regex mode (patterns, identifiers, literal text):

python3 $SKILL_DIR/scripts/search.py ./repo "def handle_error"
python3 $SKILL_DIR/scripts/search.py ./repo "class.*Exception" --regex
python3 $SKILL_DIR/scripts/search.py ./repo "TODO|FIXME|HACK"

Semantic mode (concepts, natural language):

python3 $SKILL_DIR/scripts/search.py ./repo "retry logic with backoff" --semantic
python3 $SKILL_DIR/scripts/search.py ./repo "authentication flow"
python3 $SKILL_DIR/scripts/search.py ./repo "error handling strategy"

Auto-detection: short queries and code-like tokens → regex. Multi-word natural language → semantic. Override with --regex or --semantic.

Binding-resolved mode (Python only — pyright via the python-lsp skill):

python3 $SKILL_DIR/scripts/search.py ./repo --refs SYMBOL    # find all real uses
python3 $SKILL_DIR/scripts/search.py ./repo --def SYMBOL     # go-to-definition
python3 $SKILL_DIR/scripts/search.py ./repo --hover SYMBOL   # inferred type/signature

Regex mode matches text, so a cross-reference for a function false-positives on shadowed and same-named-but-unrelated symbols. --refs is binding-resolved: pyright excludes the unrelated same-named symbol and follows imports. Use it when you need a true "find all callers/users" for a .py symbol, not a text grep.

The tier is engaged lazily — pyright's index cost is paid only when you ask for --refs/--def/--hover, never on ordinary searches. It is Python-only; for non-.py sources, or when pyright/node is unavailable, it prints a one-line degradation note and falls back to the regex text path. Each takes a single bare symbol name and is mutually exclusive with the other two and with text queries.

Options

  • --regex / --semantic: Force search mode
  • --refs SYMBOL / --def SYMBOL / --hover SYMBOL: Binding-resolved Python queries via pyright (see Binding-resolved mode above)
  • --expand: Return full function bodies via tree-sitting AST context
  • --benchmark: Compare indexed regex vs brute-force ripgrep
  • --branch NAME: Git branch for GitHub URLs (default: main)
  • --skip DIRS: Comma-separated directories to skip
  • --json: Machine-readable output
  • -v: Show index stats and query routing decisions

How It Works

Regex search builds a sparse n-gram inverted index over all files. Queries are decomposed into literal fragments, looked up in the index to identify candidate files (typically 90-99% reduction), then verified with ripgrep. Frequency-weighted n-grams make rare character sequences more selective.

Semantic search builds a TF-IDF index over code chunks (functions, classes, structural entries). Queries are ranked by cosine similarity.

Context expansion (--expand) uses tree-sitting's AST cache to identify function/class boundaries, returning complete structural units rather than line fragments. On first use, tree-sitting scans the repo (~700ms for 250 files); subsequent expansions are sub-millisecond.

Small codebases (< 20 files) skip indexing entirely — direct ripgrep is faster when there's nothing to narrow.

Mixed Queries

Multiple queries can use different modes in a single invocation. Each query is auto-routed independently, and indexes are built once per mode:

python3 $SKILL_DIR/scripts/search.py ./repo \
  "class.*Error" \
  "error recovery strategy" \
  "def retry"

Dependencies

  • tree-sitting: Provides AST context expansion for --expand and the symbol→position resolution that seeds the binding-resolved tier (--refs/--def/--hover). Auto-installs the bare tree-sitter package when either is used (grammars are bundled). Regex and semantic search work without it.
  • ripgrep: Required for regex verification. Install via uv tool install ripgrep.
  • scikit-learn: Required for semantic mode. Installs automatically.
  • python-lsp: Provides the binding-resolved tier (--refs/--def/--hover). Self-bootstraps pyright on first use and requires system node (v18+). Not required — without it those flags degrade to the regex text path.

When to Use — narrow, by design

The ONE recommended use: binding-resolved Python symbol queries.

  • "find all callers of X" / "where is X really defined" for a .py symbol, when same-named-but-unrelated symbols would pollute a text grep. Empirical basis: rg get on psf/requests returned 232 hits, 224 of them false; --refs get excluded all 224 (2026-06-15).

When NOT to Use — which is most of the time

Everything else. Measured head-to-head on real issue-localization tasks (7 scikit-learn issues with merged fix-PRs, gold = PR diff files, 2026-07-04, replicating the file-discovery metric of arXiv:2602.11988):

  • Literal tokens / identifiers: naive rg -l tied or beat the indexed tier on recall@10 in every instance, at 0.4s vs 25s.
  • Concept / natural-language search: the TF-IDF semantic tier never beat identifier grep — not even on identifier-poor issues, which are themselves rare (~0.3% of merged-PR traffic in the sample).
  • First encounter / "what is this repo": use exploring-codebases.
  • Repos under ~20 files: read them.

The self-test before invoking: would plain rg return the same answer? If yes, use rg. The indexed-regex and semantic tiers are retained for completeness and for corpora where they may yet earn their cost (very large repos, non-code document collections), but they carry the burden of proof.

Files

  • scripts/search.py — Entry point, query routing, output formatting
  • scripts/resolve.py — Input source resolution (GitHub, uploads, archives)
  • scripts/context.py — tree-sitting-based AST context expansion
  • scripts/ngram_index.py — Sparse n-gram inverted index, regex decomposition
  • scripts/sparse_ngrams.py — Core n-gram algorithms, frequency weights
  • scripts/code_rag.py — TF-IDF semantic search over code chunks
  • scripts/lsp_refs.py — Binding-resolved Python tier: symbol→position resolution (tree-sitting), pyright queries (python-lsp), soft fallback