Guide · Agent skill
How to use the Fine-Tuning Expert agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The fine-tuning-expert skill is an agent skill that makes an AI coding agent act as a guide for customizing an existing language model on your own data. It is written for ML engineers and developers who already know some Python and want a structured workflow for LoRA or QLoRA fine-tuning. After this guide you will know how to install the skill, how to ask for help with it, and what you need on your machine before you try a training run.
Fine-Tuning ExpertJeffallan/claude-skills · MIT Download the skill Quick start
- 1Download the skill ZIP. Get the fine-tuning-expert ZIP from this page.
- 2Unzip into a skills folder. Unzip it into ~/.claude/skills/ for personal use, or into .claude/skills/ inside a project.
- 3Or upload it in the Claude apps. In the Claude apps, go to Settings → Capabilities → Skills and upload the skill.
- 4Prepare your environment. Install Python with transformers, datasets, peft and trl, plus PyTorch, and bitsandbytes if you plan to use QLoRA.
- 5Get access to the base model. Create a Hugging Face account and accept the licence of any gated model you plan to use.
- 6Ask for help in plain language. Describe your model, data and GPU, and mention a trigger term such as LoRA or fine-tuning.
- 7Validate, train and evaluate. Check your JSONL dataset first, then train, then compare the result against the base model before merging.
What does the fine-tuning-expert skill make the agent do?
When you ask for help adapting a model, the skill has the agent follow a set workflow: check and format your training data, pick a lightweight method such as LoRA or QLoRA, set training settings, compare the result against the original model, and merge and optimize it for serving.
The skill also supplies example Python code using Hugging Face libraries, along with reference notes on each stage. It covers dataset preparation, LoRA/PEFT, hyperparameter tuning, evaluation metrics, and deployment optimization.
- Dataset preparation: validate and format data before training starts
- Method selection: LoRA for most tasks, QLoRA when GPU memory is limited
- Training: configure hyperparameters and watch validation loss
- Evaluation: compare the fine-tuned model against the base model
- Deployment: merge adapter weights, quantize, and measure inference
How do you install the fine-tuning-expert skill?
Download the ZIP for the skill from our page and unzip it into one of two locations. For a personal setup, use ~/.claude/skills/. For a single project, use the .claude/skills/ folder inside that project.
If you use the Claude apps instead, you can upload the skill under Settings → Capabilities → Skills.
- Personal install: unzip into ~/.claude/skills/
- Project install: unzip into .claude/skills/ in your project folder
- Claude apps: upload the skill under Settings → Capabilities → Skills
How do you trigger the skill with a natural request?
You do not need a special command. The skill's description lists trigger terms, so ask in plain language that mentions the task. Good trigger terms include LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, and custom model.
Be specific about your model, data and hardware. The more context you give, the more useful the agent's plan will be.
- "Help me set up a LoRA fine-tune of a 7B model on my support tickets stored as JSONL."
- "Check whether my instruction dataset is ready for QLoRA training."
- "What LoRA rank and learning rate should I start with for a small classification task?"
- "Show me how to merge my LoRA adapter into the base model for deployment."
What does a typical LoRA example look like?
The skill's minimal working example loads a base model, configures a LoRA adapter with Hugging Face PEFT, loads an Alpaca-style JSONL dataset, sets training arguments, trains with SFTTrainer, and saves the adapter weights. The example uses a gated Llama model, so you need a Hugging Face account and an accepted licence for that model.
The QLoRA variant adds a 4-bit quantization configuration before the model is loaded. The skill also includes a snippet for merging the adapter into the base model for deployment.
- Rank r=16 and lora_alpha=32 are the example settings; the comments note that alpha is typically 2× rank
- The example uses a learning rate of 2e-4 with cosine scheduling and a warmup ratio of 0.03
- The adapter is saved to ./lora-adapter; the merged model is saved to ./merged-model
model = get_peft_model(model, lora_config)
model.print_trainable_parameters() # verify: should be ~0.1–1% of total paramsWhat do you need before you start?
You need a Python environment with the Hugging Face transformers, datasets, peft and trl libraries, plus PyTorch and a GPU suited to your model size. Memory needs vary by model size. For the QLoRA 4-bit variant you also need bitsandbytes.
You need access to the base model weights on Hugging Face. The example uses a gated Llama model, which requires an account and an accepted licence. The skill references a validate_dataset.py script that is not included in the listed files, so you would need to write or supply it yourself.
- Python with transformers, datasets, peft and trl
- PyTorch with a suitable GPU
- bitsandbytes for QLoRA
- A Hugging Face account and access to the base model weights
- A validate_dataset.py script of your own, since it is not included
What safety points should you check?
The skill does not run scripts on its own. It does use the network: the example code downloads models and datasets from Hugging Face, so expect network access during training setup.
The skill does not check licences. Confirm that the base model's licence or terms allow fine-tuning and allow you to redistribute derived weights before you start. The example code and advice are generic, so check them against your hardware, licences and data before any large or production run.
- Scripts: none are run by the skill itself
- Network: yes, for downloading models and datasets
- Licences: check the base model's terms yourself; the skill does not verify them
- Review the generic example code before running it at scale
What tips help you get better results?
The skill's own rules are useful habits. Validate dataset quality before training, use warmup, watch training and validation loss, and version your datasets and checkpoints. If validation loss plateaus or rises, the skill treats that as a sign of overfitting.
Evaluate before you deploy. Compare the fine-tuned model against the base model on a held-out set and check latency. Use regularisation such as dropout and weight decay, and early stopping on small datasets.
- Use parameter-efficient methods such as LoRA for large models (above 7B)
- Include a learning rate warmup in every run
- Document hyperparameters and training config
- Do not merge adapters with mismatched rank, base model or target modules
- Do not deploy without evaluation on a held-out set and a latency benchmark
What people use it for
- Adapting an open-weight language model to answer questions in your company's support style
- Preparing an Alpaca-style JSONL instruction dataset and checking it for errors and duplicates
- Running a LoRA or QLoRA fine-tune on a single GPU with memory limits
- Comparing a fine-tuned model against its base model on a held-out test set before deployment
- Merging a LoRA adapter into the base model to produce a standalone model for serving
Frequently asked questions
How do I install the fine-tuning-expert skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into .claude/skills/ in a project folder. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.
Is the fine-tuning-expert skill free?
The skill is released under the MIT licence, as listed on this page. The base models you fine-tune have their own licences, which the skill does not check, so review those separately.
What do I need to run the fine-tuning-expert examples?
You need Python with transformers, datasets, peft and trl, PyTorch and a suitable GPU. QLoRA also needs bitsandbytes, and the example model is gated, so you need a Hugging Face account and an accepted licence.
Does the skill run scripts or use the internet?
The skill does not run scripts itself, but it uses the network, for example to download models and datasets from Hugging Face. It references a validate_dataset.py script that is not included, so you must write or supply it.
Can a beginner use this skill?
The skill assumes ML experience. If you only want to use an existing chat model without training it, you do not need this skill. It is better suited to ML engineers and developers who want a structured LoRA or QLoRA workflow.
How does it compare with other fine-tuning guides or skills?
This page does not compare it with other options. The skill's strength is its structured workflow from data checks through evaluation and deployment, but its example code is generic and needs checking against your own hardware, licences and data.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are an ML engineer or developer who wants a structured starting point for LoRA or QLoRA fine-tuning, with dataset checks and base-model comparison built in. Pick something else if you only want to chat with an existing model, or if you need production training without expert review, since the example code is generic and the skill does not check model licences.
Download the skill