Guide · Agent skill
How to use the Airflow DAG Patterns agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The airflow-dag-patterns skill is a set of instructions that teaches an AI agent established best practices for writing Apache Airflow DAGs, which are scheduled sets of tasks that move and process data. It is aimed at data engineers maintaining ETL pipelines, developers new to Airflow, and teams reviewing existing DAGs. After this guide you will know how to install the skill, trigger it with plain requests, and what to check before running the code it produces.
Airflow DAG Patternswshobson/agents · MIT Download the skill Quick start
- 1Download the ZIP. Download the airflow-dag-patterns ZIP from this page.
- 2Unzip the skill. Unzip the folder containing SKILL.md and references/details.md.
- 3Place the skill folder. Move it to ~/.claude/skills/ for personal use or to .claude/skills/ in your project.
- 4Or upload it in the Claude apps. Go to Settings → Capabilities → Skills and upload the ZIP.
- 5Confirm you have Airflow and Python. Make sure your environment has Apache Airflow (2.x APIs) and Python available.
- 6Ask for a DAG in plain language. Describe the pipeline, schedule and data sources, and mention Airflow.
- 7Review and test the output. Check schedule, catchup, retries and paths, then run unit and integration tests before production.
What does the airflow-dag-patterns skill make the agent do?
When the skill is installed, the agent follows patterns for DAG design, operators, sensors, testing and deployment when you ask it to build or debug a pipeline. The skill's core principles are that tasks should be idempotent (running twice produces the same result), atomic (tasks succeed or fail completely), incremental (process only new or changed data) and observable (logs, metrics and alerts at every step).
The skill also covers task dependency patterns such as linear chains, fan-out, fan-in and combined layouts, and a list of do's and don'ts such as avoiding depends_on_past=True and hardcoding dates.
- Use the TaskFlow API for cleaner code and automatic XCom handling
- Set timeouts to prevent zombie tasks
- Use mode='reschedule' on sensors so they free up workers
- Use {{ ds }} macros instead of hardcoded dates
- Keep heavy logic out of the DAG file and import it from modules
How do you install the airflow-dag-patterns skill?
Download the ZIP for the skill from this page and unzip it. For a personal install, place the unzipped folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.
You can also upload the ZIP in the Claude apps under Settings → Capabilities → Skills.
- Personal: ~/.claude/skills/
- Project: .claude/skills/ inside your project folder
- Claude apps: Settings → Capabilities → Skills, then upload the ZIP
What do you need before using it?
The skill provides guidance and example code, not a running Airflow environment. The code examples use the Airflow 2.x DAG and operator APIs, so you need Apache Airflow and Python on the machine or server where the DAGs will run. The skill does not run scripts and does not use the network.
- Apache Airflow (the examples use the Airflow 2.x DAG and operator APIs)
- Python
How do you trigger the skill with a natural request?
You do not need a special command. Describe the pipeline task in plain language and mention Airflow. The skill is relevant when you are creating data pipelines, orchestrating workflows, scheduling batch jobs, designing DAG dependencies, writing sensors or operators, testing DAGs locally, or debugging failed runs.
Be specific about what the pipeline does, how often it runs and what it reads or writes. The more context you give, the more the generated DAG can follow the conventions in the skill.
- Build a daily Airflow DAG that extracts a CSV from S3, cleans it and loads it to a warehouse table.
- Review this DAG for retry, catchup and dependency problems.
- Write a sensor that waits for a file to land, without holding a worker slot.
- Help me debug a failed Airflow run and suggest safer retry settings.
What does a typical DAG look like?
The skill's quick start shows a daily ETL DAG with retries, exponential backoff, a single active run, catchup disabled and a simple start, extract and end sequence. The reference file adds a TaskFlow example that uses the @dag and @task decorators and passes data between tasks.
Treat generated code as a starting point. Check the schedule, start_date, catchup setting and any file paths against your own environment before deploying.
start >> extract >> endWhat safety notes should you know?
The skill runs no scripts and uses no network access, so installing it does not execute code on your machine. However, the DAGs the agent writes are real Python that Airflow will run, so they can read, write and delete data. Review them before use, and test them in a non-production environment.
The skill's own guidance also says generated DAGs still need review and testing before production use.
- Review every generated DAG before deploying it
- Test locally with unit and integration tests, as the skill recommends
- Check file paths, bucket names and warehouse targets carefully
What tips help you get better results?
Ask the agent to explain why it chose a pattern, such as a sensor in reschedule mode or a particular retry setting, so you can judge it. If you have an existing DAG, paste it in and ask for a review against the retry, catchup and dependency guidance.
The skill is most useful as a checklist for review. Use it alongside Airflow's own documentation for the exact parameters of your installed version.
- Paste existing DAG code for a targeted review
- Name your Airflow version so the agent uses the right API
- Ask for a test file alongside the DAG
What people use it for
- Building a new daily ETL DAG with retries and a single active run
- Reviewing an existing DAG for retry, catchup and dependency issues
- Replacing a worker-blocking sensor with one in reschedule mode
- Converting Python task functions to the TaskFlow API
- Planning local tests for a batch pipeline before deployment
Frequently asked questions
Is the airflow-dag-patterns skill free?
The skill is published under the MIT licence, which is a permissive open-source licence. Check the licence file in the source repository for the full terms.
What do I need to use the airflow-dag-patterns skill?
You need Apache Airflow and Python. The code examples use the Airflow 2.x DAG and operator APIs. The skill itself runs no scripts and uses no network.
How do I install an agent skill like this one?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder. You can also upload the ZIP in the Claude apps under Settings → Capabilities → Skills.
Does the skill deploy or run my Airflow pipelines?
No. It provides guidance and example code only. It does not run Airflow or deploy DAGs, so the generated code must be reviewed and tested by you before production use.
Can I use this skill with tools other than Airflow?
The patterns are specific to Apache Airflow, so the skill is not useful for projects that do not use it. For other orchestrators you would need guidance written for those tools.
Why does the skill say to avoid depends_on_past=True and catchup?
The skill lists depends_on_past=True as a pattern to avoid because it creates bottlenecks, and says not to skip catchup blindly without understanding its implications. Ask the agent to explain the trade-offs for your specific schedule.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
This skill suits data engineers and newcomers who want Airflow DAGs that follow common production conventions, and teams that want a checklist for reviewing retries, catchup and dependencies. It is not suitable if you do not use Apache Airflow or expect it to deploy and run pipelines for you. Treat its output as reviewed-before-use example code.
Download the skill