Skip to content
Get packs

Quantized Model Export

Export a promoted fine-tuned model into a deployment format and smoke-test it after export.

Source Download skill
Repository
wshobson/agents
Stars
42,095
Licence
MIT
Files
2 files

What it does

This skill guides an AI agent through exporting a fine-tuned language model that has already passed a promotion gate into a format suited to its target hardware, such as FP8, AWQ INT4, or GGUF. It explains which format to pick for a given GPU or edge device, warns about workloads like long-context, code, and math that break under INT4, and requires a smoke test comparing outputs before and after export. The skill also points to a reference file with runnable command sequences.

Great for

  • ML engineers deploying a fine-tuned model to a datacenter GPU, a older GPU, or a local llama.cpp setup
  • Teams that already run a checkpoint-promotion and eval-harness workflow and need a consistent export step
  • Anyone choosing between merged and LoRA-only artifacts for serving

Skip it if

  • People without a trained or fine-tuned checkpoint, since the skill assumes a promoted model as input
  • Teams targeting NVFP4 on GB10 hardware, which the skill says is slower than FP8 and should be avoided
  • Anyone expecting a file-existence check to prove an export works, since the skill states that malformed exports still load

Try asking

  • “Export my promoted checkpoint to FP8 for an H100 serving stack with vLLM.”
  • “Which quantization format should I use for a fine-tuned model running on a GB10 laptop?”
  • “My exported model is giving garbled output after quantization, help me run the smoke test.”
  • “Create a LoRA-only export of this adapter and check it against the golden prompts.”
Step-by-step guide: how to use Quantized Model ExportInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 2 original files, the MIT licence and an attribution note.

    quantized-export.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip quantized-export.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip quantized-export.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Provide the exact GPU class, serving stack, and whether long-context, code, or math workloads are in scope so the agent picks the right format.
  • Persist the pre-export outputs with fixed seeds and greedy decoding so the post-export comparison is meaningful.
  • Re-run the smoke test after any quantization method or runtime version change, not just the first export.

What's inside

  • references/export-commands.md
  • SKILL.md

Redistributed unmodified from wshobson/agents under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.