verl RL Training for LLMs
Guides an agent through reinforcement learning post-training of LLMs with verl (GRPO, PPO and more).
- Repository
- Orchestra-Research/AI-Research-SKILLs
- Stars
- 14,195
- Licence
- MIT
- Files
- 3 files
What it does
Once installed, the agent can give step-by-step guidance for training large language models with reinforcement learning using the verl library from ByteDance's Seed team. It covers choosing an algorithm such as GRPO or PPO, preparing a dataset and reward function, writing a YAML training config, launching single-node or multi-node Ray jobs, and picking a backend like FSDP, Megatron-LM, vLLM or SGLang. It is a guide for a specialist workflow, not a tool that trains models by itself.
Great for
- ML engineers setting up RLHF or GRPO post-training runs on GPU clusters
- Researchers comparing RL algorithms such as PPO, GRPO, and DAPO for reasoning models
Skip it if
- Simple supervised fine-tuning or DPO, which the skill itself says is better served by TRL or Axolotl
- Beginners without GPU access, since the workflows assume 8+ GPUs and multi-node clusters; following them can incur large cloud costs
Try asking
- “Set up a GRPO training run on GSM8K with Qwen2.5-7B using verl”
- “Write a reward function and parquet dataset for math reasoning with verl”
- “Switch my verl PPO config from GRPO to GAE with a critic model”
- “Configure a Megatron backend with tensor and pipeline parallelism for a 70B model”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 3 original files, the MIT licence and an attribution note.
verl.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip verl.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip verl.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Start with a small model and a few GPUs to validate the config before scaling to a full cluster.
- Check the reward is increasing and run held-out evaluation, as the skill's monitoring checklist suggests.
- Pin library versions to those listed in the dependencies, since APIs differ between verl releases.
What's inside
- references/api-reference.md
- references/troubleshooting.md
- SKILL.md
Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.