Agent-Skills.md

Agent Skills: Custom Data Deep Research

Build and validate custom-corpus inputs for deep research from Google Docs, YouTube, local files, S3, arXiv, databases, Slack, email, tickets, and similar non-web sources. Use when preparing private or domain-specific source manifests, validating source safety, or turning custom data into cited research inputs.

UncategorizedID: closedloop-technologies/awesome-deep-researchers/deep-research-custom-data

Repository

closedloop-technologies/awesome-deep-researchers
closedloop-technologiesLicense: MIT
71

Install this agent skill to your local

pnpm dlx add-skill https://github.com/closedloop-technologies/climbhill/tree/HEAD/.agents/skills/deep-research-custom-data

Skill Files

Browse the full folder contents for deep-research-custom-data.

Download Skill

Loading file tree…

.agents/skills/deep-research-custom-data/SKILL.md

Skill Metadata

Name
deep-research-custom-data
Description
Build and validate custom-corpus inputs for deep research from Google Docs, YouTube, local files, S3, arXiv, databases, Slack, email, tickets, and similar non-web sources. Use when preparing private or domain-specific source manifests, validating source safety, or turning custom data into cited research inputs.

Custom Data Deep Research

Use this skill when the research corpus is not just the open web. The goal is to turn private, semi-structured, or domain-specific sources into a cited corpus that a deep research agent can inspect safely.

Workflow

  1. Define the research question and allowed sources.
  2. Export or read sources with read-only credentials.
  3. Convert each source into markdown or JSON plus metadata.
  4. Build an evidence manifest with stable IDs and offsets.
  5. Run the research agent over the allowed corpus.
  6. Normalize the report, findings, uncertainties, and method into OKF.

Source Guidance

| Source | How to Use | Citation Anchor | | --- | --- | --- | | Google Docs / Drive | Search/read Drive files, export Docs, preserve file IDs and revision times. | file ID, title, URL, heading, revision timestamp | | YouTube | Pull transcript and metadata; prefer official captions. | video URL, channel, upload date, timestamp range | | Local files | Crawl only allowlisted paths, hash files, convert PDFs/docs/text. | relative path, sha256, modified time, page/line/heading | | S3 | Use read-only IAM, sync scoped prefixes, save object manifest. | s3://bucket/key, version ID, ETag, row/page/byte offset | | arXiv | Fetch metadata and PDFs by arXiv ID and version. | arXiv ID/version, authors, section/page | | Databases / CSV | Export query, schema, row counts, and sampled/aggregate evidence. | query ID, table, primary key, aggregate definition |

Prompt Contract

Research question:
<question>

Allowed corpus:
<source IDs or OKF bundle paths>

Rules:
- Use only allowed sources unless public web supplementation is explicitly requested.
- Cite source ID plus exact anchor for every factual claim.
- Separate directly supported findings from inferences.
- Record missing, conflicting, stale, or low-confidence evidence.
- Return an OKF-compatible report.

Manifest Validation

Create a JSON manifest before indexing custom data. Validate it with:

python .agents/skills/deep-research-custom-data/scripts/validate_manifest.py \
  staging/custom-corpus/manifest.json

The validator checks corpus metadata, supported source types, stable source IDs, and citation anchors before a research agent uses the corpus.

Use docs/examples/custom-data-corpus.example.json as the minimal fixture for benchmark and CI validation.

OKF Output

Use concept files for source records and research outputs:

  • sources/<source-id>.md with type such as Google Drive Document, YouTube Transcript, Local File Evidence, S3 Object Evidence, or Scholarly Paper.
  • claims/claim-register.md with type Evidence Claim Register.
  • method.md with type Research Method.
  • report.md, findings.md, and uncertainties.md for the final research output.

See docs/custom-data-sources.md for detailed extraction and governance guidance.

Safety

  • Confirm authorization before accessing private data.
  • Use read-only credentials and scoped folders, prefixes, or queries.
  • Do not commit raw private extracts unless the repository is approved for that data.
  • Preserve retention/deletion requirements in method.md.