Guide · Agent skill
How to use the Dataset Curation agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The dataset-curation skill is a set of instructions that guides an AI agent through preparing data for fine-tuning a language model: choosing the row format for your training method, applying the model's chat template correctly, packing short examples, and adding synthetic data safely. It is for engineers and teams who already have a fine-tuning method in mind and need clean, documented training data. After this guide you will know how to install the skill, how to ask for it in plain language, and what it expects you to produce before a training run starts.
Dataset Curationwshobson/agents · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the ZIP from this page and unzip it so the dataset-curation folder contains SKILL.md and the references folder.
- 2Place the folder. Move the folder into ~/.claude/skills/ for personal use, or into the project's .claude/skills/ folder for one project. You can also upload it under Settings → Capabilities → Skills in the Claude apps.
- 3Choose your method. Confirm the fine-tuning method first, using finetuning-method-selection, because this skill starts after that decision.
- 4Ask for curation in plain language. Describe your raw data and target method, for example asking the agent to convert transcripts into ChatML messages for SFT.
- 5Apply templates and masking. Have the agent apply the model's chat template before packing and mask loss to assistant turns only.
- 6Inspect packed sequences. Decode and read 5 to 10 packed sequences to confirm boundaries, template markers and the assistant-only loss mask.
- 7Complete the dataset card. Record provenance, counts, synthetic/real ratio, dedup method, template and packing settings before training.
What the dataset-curation skill makes the agent do
An agent skill is a folder of instructions that an AI agent loads when a task matches the skill's description. This one covers the data preparation stage of fine-tuning. It is not a method chooser: it assumes a separate finetuning-method-selection skill has already decided that fine-tuning is the right approach.
- Pick the row format that matches the training method, such as instruct pairs, ChatML conversations, preference pairs, or prompt-only rows.
- Apply the target model's chat template before any concatenation or packing.
- Mask the loss so only assistant responses contribute to training.
- Pack short examples into longer sequences and check the packed output by decoding it.
- Mix in synthetic (model-generated) data while keeping at least 25% real data.
- Write a dataset card recording provenance, counts, synthetic/real ratio, dedup method, template and packing settings.
Install the skill from the ZIP file
Download the ZIP for the skill from this page and unzip it. The skill folder contains SKILL.md and two reference files, references/formats-and-templates.md and references/synthetic-data.md. Put the folder in one of the following places, depending on how you want to use it.
You can also upload the skill in the Claude apps through Settings → Capabilities → Skills.
- Personal install: unzip into ~/.claude/skills/ so it is available across your projects.
- Project install: unzip into the project's .claude/skills/ folder so it is available only in that project.
Trigger the skill with natural requests
You do not need a special command to use the skill. The agent reads the skill's description and loads it when your request matches. The description covers converting raw data into training format, applying chat templates, configuring sequence packing, generating synthetic training data, and writing a dataset card before a run.
Name the task and the data you have. Good requests say what the input is and what training method you plan to use.
- "Convert these support transcripts into ChatML messages for SFT."
- "Apply the model's chat template to this dataset and mask loss to assistant turns."
- "Pack these examples into 2048-token sequences and show me a few to check."
- "Generate synthetic examples for this task and filter them, keeping at least 25% real data."
- "Write the dataset card for this set before we start training."
Example workflow
A typical session starts with raw examples and a routing decision from finetuning-method-selection. The agent chooses the format from the method, templates and masks the data, packs it, and then asks you to inspect a sample of packed sequences. The skill names this inspection as mandatory, because packing bugs do not show up in the loss curve and only appear later in evaluation.
The skill also describes a sanity check for masking: decode only the unmasked positions and confirm that only assistant text remains.
for seq in packed_dataset.select(range(10)):
print(tokenizer.decode(seq["input_ids"]))What you need before you start
The skill assumes a working Python environment with a tokenizer library and the target model's tokenizer. The examples use Hugging Face style tokenizers and datasets. If you follow the trainer-based masking approach, you need TRL, and the skill specifies the current version for assistant_only_loss.
The skill refers to sibling skills (finetuning-method-selection, preference-optimization, grpo-rlvr-training, trace-to-training-data, eval-harness-first) and a /finetune command. These are not included in the files for this skill, so the agent cannot rely on them unless you have them separately.
- Python with a tokenizer library and the target model's tokenizer.
- TRL (current version) if you use trainer-based masking.
- Access to the sibling skills and /finetune command if you want the full workflow.
- The skill runs no scripts and uses no network access.
Safety notes on scripts and network use
The skill's files contain instructions and code sketches, not executable scripts. It does not use the network. Any code the agent writes or runs in your environment is still code you should review before running, especially when it touches your data files or model tokenizers.
- Review the code sketches before running them against real data.
- Keep copies of your raw data; the skill transforms it into training format.
- Do not treat the numeric guidance as fixed. Row floors, accept rates and efficiency multipliers are rules of thumb and should be checked against your own model and data.
Tips for better dataset results
Most of the value in this skill comes from the checks it enforces. Treat the dataset card as a gate you fill in during preparation, not a summary written afterward.
- Use one key pair consistently across the dataset for instruct rows.
- Prefer a smaller verified, deduplicated set over a larger noisy one.
- Plan for synthetic volume: the skill notes typical accept rates of 10–30% after filtering, so a 10,000-row target at 15% accept needs roughly 65,000 raw generations.
- Use the same chat template at training, inference and evaluation time.
- Check the Phase 2 exit checklist before handing off to training.
What people use it for
- Converting customer support transcripts into ChatML messages for multi-turn SFT.
- Reformatting instruction and response pairs into a consistent instruct format before training.
- Preparing preference pairs with prompt, chosen and rejected fields for a preference training run, with the method details handled by a separate skill.
- Generating synthetic examples for a narrow task and filtering them while keeping a real-data floor.
- Writing an auditable dataset card for a team that needs a record before launching a training run.
Frequently asked questions
Do I need to install anything to use the dataset-curation skill?
You need to place the skill folder in ~/.claude/skills/ or a project's .claude/skills/ folder, or upload it in the Claude apps under Settings → Capabilities → Skills. For the work itself, the skill expects Python with a tokenizer library and the target model's tokenizer, and TRL if you use trainer-based masking.
Is the dataset-curation skill free?
The source lists the skill's licence as MIT. It does not state any price, so check the directory page for current terms of use.
Does this skill choose the fine-tuning method for me?
No. The skill assumes routing from finetuning-method-selection and will not help choose the method. Use that skill first, then use this one to prepare the data.
How many rows do I need for SFT?
The skill recommends about 1,000 or more rows as a floor for SFT, not a target. Above that floor it prioritises quality over quantity, so a smaller verified and deduplicated set is preferred to a larger noisy one.
Why does the skill insist on decoding packed sequences?
Packing bugs are silent, so the loss curve can look normal while eval quality suffers later. The skill makes decoding and inspecting 5 to 10 packed sequences mandatory before a full run.
What is the 25% real data rule?
The skill says to keep at least 25% real data in the final mix as a guard against quality collapse from training on model-generated data. General-domain replay rows can count toward this floor, and the dataset card should state which rows count as real.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are an engineer or team preparing instruction, chat or preference data and want a checked, documented dataset before training. Pick something else if you have not chosen a fine-tuning method yet, or if you only need prompt engineering or one-off chat, where fine-tuning adds effort without benefit.
Download the skill