LLM Fine-Tuning Expert
Guides an agent through LoRA/QLoRA fine-tuning: dataset prep, hyperparameters, evaluation, and adapters.
- Repository
- RightNow-AI/openfang
- Stars
- 18,243
- Licence
- Apache-2.0
- Files
- 1 file
What it does
This skill gives the agent a expert playbook for adapting an existing large language model to a specific task or domain without retraining the whole model. It covers preparing instruction-response training data, choosing LoRA or QLoRA settings such as rank, learning rate and epochs, evaluating results beyond simple loss scores, and saving or merging the resulting small adapter files. It is guidance and know-how only; the file itself does not run any training.
Great for
- Machine learning engineers who want an assistant to plan or review a LoRA or QLoRA fine-tuning run
- Teams preparing instruction-response datasets for a domain model such as legal or medical text
- Developers learning the Hugging Face PEFT workflow who want step-by-step guidance
Skip it if
- Non-technical users who only want to use an existing AI chatbot better; this is about modifying model weights, which requires GPUs and ML tooling
- Work involving sensitive personal, medical or copyrighted data without a review process, since the skill warns against training on such data unfiltered and the agent could otherwise help move it into a training pipeline
Try asking
- “Plan a QLoRA fine-tune of a 7B model on 2,000 customer-support transcripts and list the hyperparameters to start with”
- “Review my training dataset format and tell me how to structure instruction, system and response fields”
- “Explain why my fine-tuned model's validation loss is dropping but its outputs look repetitive, and what to change”
- “Write a PEFT training script outline that loads a base model, applies a LoRA config and merges the adapter for inference”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 1 original files, the Apache-2.0 licence and an attribution note.
llm-finetuning.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip llm-finetuning.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip llm-finetuning.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Name your base model, hardware and dataset size up front, since the advice on ranks and learning rates depends on them
- Ask the agent to produce code and then run it yourself in a suitable environment; the skill itself does not execute anything
- Keep a held-out evaluation set and check for overlap with training data as the skill recommends
What's inside
- SKILL.md
Redistributed unmodified from RightNow-AI/openfang under the Apache-2.0 licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.