Guide · Agent skill
How to use the miles RL Training agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The miles RL Training skill gives an AI agent a detailed playbook for reinforcement learning (RL) post-training of very large mixture-of-experts (MoE) models using miles, a production fork of the slime framework. It is written for ML engineers who work on multi-GPU data-center hardware and want a guided starting point for FP8 or INT4 training, train-inference alignment and speculative RL. After this guide you will know how to install the skill, how to ask for it in plain language, and what it can and cannot do for you.
miles RL TrainingOrchestra-Research/AI-Research-SKILLs · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the ZIP from our skill page and unzip it so the miles-rl-training folder contains SKILL.md and the references folder.
- 2Place it where your agent looks. Move the folder to ~/.claude/skills/ for personal use or to .claude/skills/ in your project. Alternatively, upload the ZIP in the Claude apps under Settings → Capabilities → Skills.
- 3Check your environment. Confirm you have NVIDIA GPUs (H100 or H200 recommended), Docker with NVIDIA GPU support or a source install of miles, and the required Python packages.
- 4Ask a specific request. Describe your model, GPU count and goal, such as post-training Qwen3-MoE with FP8 on 8 H100s, so the agent loads the skill and produces relevant guidance.
- 5Review and run the commands yourself. Read the launch commands and flags the agent writes, replace placeholder paths with your own, and run them on your cluster. The skill does not run training for you.
What the miles skill makes the agent do
A skill is a folder of instructions that an agent loads when a task matches its description. The miles-rl-training skill is built around a SKILL.md file and two reference files, one for the API and one for troubleshooting. When it is active, the agent can explain when miles is a better choice than alternatives, describe how to set up FP8 or INT4 low-precision training, explain how Rollout Routing Replay (R3) keeps expert routing consistent between inference and training, and walk through enabling EAGLE speculative decoding.
The agent can then draft launch commands and configuration flags for these workflows. The skill itself does not run training, so you still launch every job yourself on your own cluster.
- Compares miles with slime, verl and torchforge and lists when to choose each
- Covers unified FP8, INT4 quantization-aware training and R3 for MoE models
- Explains speculative RL with EAGLE and optional online MTP training
- Provides troubleshooting notes for FP8 training collapse and train-inference mismatch
How to install the miles skill
Option one is a manual install. Download the ZIP from our skill page and unzip it into your personal skills folder at ~/.claude/skills/, or into a project-specific folder at .claude/skills/ inside your project. The skill folder should contain SKILL.md along with the references folder.
Option two is the Claude apps. Open Settings, go to Capabilities, then Skills, and upload the ZIP there. Use whichever option matches how you work; the skill behaves the same once it is loaded.
- Personal install: ~/.claude/skills/
- Project install: .claude/skills/ inside your project folder
- Claude apps: Settings → Capabilities → Skills, then upload the ZIP
What you need before you start
The skill assumes serious hardware. The workflows are written for NVIDIA GPUs, with H100 or H200 recommended, and FP8 support is required for the FP8 workflow. You also need either Docker with NVIDIA GPU support or a source install of miles, plus Python with the packages listed in the skill: sglang-router at 0.2.3 or later, ray, torch at 2.0.0 or later and transformers at 4.40.0 or later.
You also need model checkpoints and training data on local paths, such as HuggingFace-format checkpoints. The skill uses the network, so plan for that when your cluster is restricted.
- NVIDIA GPUs (H100/H200 recommended; FP8 support for the FP8 workflow)
- Docker with NVIDIA GPU support, or a source install of miles
- Python with sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0
- Model checkpoints and training data on local paths
docker pull radixark/miles:latest
docker run --rm --gpus all --ipc=host --shm-size=16g \
-it radixark/miles:latest /bin/bash
git clone https://github.com/radixark/miles.git
cd miles
pip install -r requirements.txt
pip install -e .How to trigger the skill with natural requests
You do not need special syntax. The skill's description covers training large MoE models with FP8 or INT4, train-inference alignment, and speculative RL for throughput. Ask for those things in plain language and the agent should load the skill when the request matches.
Be specific about your model, hardware and goal. A request that names the model, the GPU count and the problem you are solving gives the agent enough to produce a useful plan.
- 'Set up miles to post-train Qwen3-MoE on 8 H100s with GRPO.'
- 'Help me configure FP8 training in miles and avoid NaN losses.'
- 'Which should I use for this MoE run: miles, slime or verl?'
- 'Write a miles config for EAGLE speculative decoding during RL rollouts.'
Example: a large MoE training configuration
The skill's basic workflow uses a Python entry point with GRPO as the advantage estimator, a checkpoint path, and parallelism settings. Treat the example below as a starting point to adapt, not a tested recipe for your cluster. Paths and sizes must match your own setup.
The skill also recommends two environment variables for FP8 block scaling. Set them before launching the job.
export NVTE_FP8_BLOCK_SCALING_FP32_SCALES=1
export CUDA_DEVICE_MAX_CONNECTIONS=1
python train.py \
--actor-num-gpus-per-node 8 \
--rollout-num-gpus 8 \
--hf-checkpoint /path/to/deepseek-v3 \
--advantage-estimator grpo \
--tensor-model-parallel-size 8 \
--expert-model-parallel-size 4 \
--prompt-data /path/to/data.jsonl \
--num-rollout 3000Safety notes: scripts, network and what to verify
The skill does not run scripts. It provides guidance and commands that you review and run yourself. Read every command the agent writes before executing it, especially anything that changes checkpoints, data paths or cluster resources.
The skill's performance claims are vendor-stated. Its figures, such as 25-40% faster rollout with speculative RL and 'exactly 0 KL divergence', are not independently verified in the skill files. Measure throughput and alignment on your own workload before you rely on them.
- Scripts: none run by the skill
- Network: the skill uses the network, so review what your environment allows
- Verify speedups and numerical alignment on your own model and data
Tips for getting good results
If FP8 training collapses, with loss exploding or NaN values, the troubleshooting reference suggests block scaling, enabling R3 for MoE models, lowering the learning rate (for example from 1e-6 to 5e-7), and warming up from BF16. For speculative drift, where acceptance rate falls over time, the skill suggests enabling online MTP training.
Some features are described conceptually and their exact flags may vary. The skill itself says to consult the miles repository for the latest configuration, so check that before you copy flags into a production job.
- Start with BF16 to confirm your pipeline works before enabling FP8
- Check that routing decisions are consistent and that loss has no NaN or Inf values
- Note that online MTP training needs a torch dist checkpoint with MTP weights, created with --mtp-num-layers 1 during conversion from HuggingFace
What people use it for
- Choosing between miles, slime, verl and torchforge for a large MoE post-training project
- Drafting a GRPO launch configuration with tensor and expert parallelism for DeepSeek V3 or Qwen3-MoE checkpoints
- Planning FP8 training with block scaling and checking the environment variables to set
- Designing an EAGLE speculative decoding setup for RL rollouts and deciding whether to enable online MTP training
Frequently asked questions
How do I install the miles RL Training skill?
Download the ZIP from our skill page and unzip it into ~/.claude/skills/ for personal use or into .claude/skills/ inside a project. In the Claude apps, you can instead upload the ZIP under Settings → Capabilities → Skills.
What hardware does the miles skill require?
The workflows assume NVIDIA GPUs, with H100 or H200 recommended, and FP8 support for the FP8 workflow. The skill is aimed at large clusters and is not suited to laptop-scale experiments.
Is the miles skill free and what is its licence?
The skill is listed under the MIT licence. The skill's files do not state a price, so check the source repository for any terms about the miles framework itself.
How does miles compare with slime, verl and torchforge?
The skill says slime is the research-grade original, verl offers flexible backend swapping, and torchforge is PyTorch-native. It recommends miles when you need FP8 or INT4 training, train-inference alignment, or speculative RL for throughput in production-style work.
Does the miles skill run training jobs for me?
No. The skill provides guidance and can write launch commands and configuration flags, but you run them yourself on your own hardware.
What should I do if FP8 training loss explodes or shows NaN values?
The troubleshooting reference suggests using blockwise scaling with NVTE_FP8_BLOCK_SCALING_FP32_SCALES=1, enabling --use-r3 for MoE models, lowering the learning rate to about 5e-7, and warming up from BF16. Check MoE routing consistency between training and inference as well.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are an ML engineer or research team post-training very large MoE models on H100 or H200 clusters and want a structured starting point for miles. Pick something else if you work on small models, have only a laptop or limited GPUs, or need verified performance numbers, since the skill's speedup and alignment claims are vendor-stated.
Download the skill