Agent-Skills.md

Agent Skills: Senior Data Engineer

Data engineering workflows for designing reliable pipelines and datasets: ingestion, transforms, orchestration, schema evolution, data contracts, quality checks, and observability. Use when building ETL/ELT, reviewing pipelines, defining warehouse/lake schemas, or diagnosing data quality incidents.

UncategorizedID: vadimcomanescu/codex-skills/senior-data-engineer

Repository

vadimcomanescu/codex-skills
vadimcomanescuLicense: MIT
218

Install this agent skill to your local

pnpm dlx add-skill https://github.com/vadimcomanescu/codex-skills/tree/HEAD/skills/.curated/data/senior-data-engineer

Skill Files

Browse the full folder contents for senior-data-engineer.

Download Skill

Loading file tree…

skills/.curated/data/senior-data-engineer/SKILL.md

Skill Metadata

Name
senior-data-engineer
Description
"Data engineering workflows for designing reliable pipelines and datasets: ingestion, transforms, orchestration, schema evolution, data contracts, quality checks, and observability. Use when building ETL/ELT, reviewing pipelines, defining warehouse/lake schemas, or diagnosing data quality incidents."

Senior Data Engineer

Make data pipelines boring: predictable, observable, and recoverable.

Quick Start

  1. Define the data contract (schema + semantics + freshness + ownership).
  2. Design the pipeline:
    • Inputs, transformations, outputs, backfills, and failure handling
  3. Data quality: checks for nulls, ranges, uniqueness, and referential integrity.
  4. Operational story: retries, checkpoints, alerting, and lineage.

Optional tool: lightweight profiling for CSV/JSONL

python ~/.codex/skills/senior-data-engineer/scripts/data_quality_scan.py path/to/data.csv --out /tmp/data_profile.json

References

  • Data contract template: references/data-contract.md
  • Pipeline checklist: references/pipeline-checklist.md