Skip to content
Get packs

Data Pipeline Expert

Guides an agent to design reliable ETL/ELT pipelines with Airflow, Spark, dbt and data quality checks.

Source Download skill
Repository
RightNow-AI/openfang
Stars
18,243
Licence
Apache-2.0
Files
1 file

What it does

When installed, this skill gives the agent a set of data engineering best practices to follow when you ask for help with data pipelines. It covers how to structure Airflow DAGs, Spark jobs, dbt projects and tests, and data quality checks, plus common patterns such as incremental loads, backfills, dead letter queues and schema evolution. It is written as guidance and does not add any tools or scripts that run on their own.

Great for

  • Data engineers who want consistent advice on building idempotent, incremental pipelines
  • Analytics engineers structuring dbt projects with staging, intermediate and mart models
  • Teams reviewing an Airflow or Spark design for reliability and observability

Skip it if

  • Running or deploying real pipelines: the skill contains only text guidance, so the agent cannot connect to your systems or verify your code works (the output still needs human review)
  • Non-technical users looking for a finished data tool or dashboard

Try asking

  • “Design an incremental Airflow DAG that loads orders from Postgres into the warehouse with retries and SLAs”
  • “Review my Spark job and suggest partitioning and broadcast join improvements”
  • “Structure a dbt project with staging, intermediate and mart models and write schema tests”
  • “Propose a dead letter queue and data quality gates for my ingestion pipeline”
Step-by-step guide: how to use Data Pipeline ExpertInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 1 original files, the Apache-2.0 licence and an attribution note.

    data-pipeline.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip data-pipeline.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip data-pipeline.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Give the agent your actual stack (warehouse, orchestrator, source systems) so advice fits your environment rather than generic defaults
  • Ask for specific artifacts such as a DAG file or dbt test YAML, then check them against your tool versions
  • Treat the recommendations as a starting checklist and validate against your team's standards before production use

What's inside

  • SKILL.md

Redistributed unmodified from RightNow-AI/openfang under the Apache-2.0 licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.