Skip to content
Get packs

LoRA & QLoRA Recipes

Configure LoRA/QLoRA fine-tuning adapters with rank, alpha, learning rate and module choices

Source Download skill
Repository
wshobson/agents
Stars
42,095
Licence
MIT
Files
3 files

What it does

When installed, the agent gets a fixed set of recommended settings for LoRA and QLoRA, two lightweight ways of training an add-on to an AI language model. It helps choose which model layers to adapt, how to set rank, alpha and learning rate, when QLoRA's memory savings are worth it, and which Unsloth or plain TRL options to use. It outputs a concrete configuration of values rather than general advice. The skill is written to be used alongside other skills for dataset preparation and training scripts.

Great for

  • ML engineers writing or reviewing a LoRA or QLoRA supervised fine-tuning config
  • Developers with a demonstration-style dataset who want sensible starting hyperparameters
  • Teams deciding between LoRA, QLoRA and full fine-tuning on limited GPU memory

Skip it if

  • Preference-based or reinforcement-learning training, since the skill assumes supervised demonstration data and points elsewhere for those methods (the routing decision is handled by another skill)
  • Users with no ML background who need a working training pipeline; the skill gives configuration values, not step-by-step setup, and assumes familiarity with Hugging Face, PEFT and GPU memory
  • Dense knowledge injection, where the skill explicitly treats full fine-tuning as the right choice and does not cover it in depth

Try asking

  • “Write a LoRA config for adapting a 7B model on my customer-support demonstrations”
  • “Review my QLoRA training script and check rank, alpha and learning rate against best practice”
  • “Should I use LoRA or QLoRA for a model that won't fit in bf16 on my 24GB GPU?”
  • “Pick target modules and rank for an SFT run on a small dataset”
Step-by-step guide: how to use LoRA & QLoRA RecipesInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    lora-qlora-recipes.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip lora-qlora-recipes.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip lora-qlora-recipes.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Give the agent your model size, GPU memory and dataset size up front so it can choose the rank row and whether QLoRA is needed.
  • Ask for the full config including SFTConfig, and check the referenced reference files for worked examples.
  • Treat the recommended values as starting points and verify them on held-out evaluation data, since the skill itself says to only raise rank when evidence shows underfitting.

What's inside

  • references/hyperparameters.md
  • references/unsloth-trl-mapping.md
  • SKILL.md

Redistributed unmodified from wshobson/agents under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.