How to use the torchforge RL Training agent skill

What the torchforge RL Training skill makes the agent do

An agent skill is a folder of instructions that an AI agent reads when your request matches the skill's purpose. This one covers torchforge, a library that separates infrastructure code (distributed training, inference and weight sync) from algorithm code, so researchers can try methods with less plumbing.

With the skill installed, the agent can walk you through setting up an environment, writing YAML training configs, defining reward functions, writing custom loss functions and scaling jobs across multiple GPUs. The skill's materials name GRPO, DAPO, CISPO, GSPO and SAPO as built-in loss functions.

• Set up a torchforge environment with the install script
• Write a YAML config for a GRPO math-reasoning run
• Define a reward function, including a custom one
• Scaffold a custom loss class and wire it into a training loop
• Launch SFT or GRPO jobs locally or with SLURM

How to install the torchforge RL Training skill

You can add the skill in one of two ways. Download the ZIP from the skill's page on our directory and unzip it into your personal skills folder, or into a project's skills folder if you want it only for one repository.

If you use the Claude apps, you can instead upload the skill under Settings → Capabilities → Skills.

• Personal install: unzip the folder into ~/.claude/skills/
• Project install: unzip the folder into .claude/skills/ inside your project
• Claude apps: Settings → Capabilities → Skills, then upload the skill

What you need before you start

The skill is for hardware-heavy work. Its workflows need several NVIDIA GPUs: at least 2 for SFT and at least 3 for GRPO. AMD users can use the ROCm install path instead. The skill also expects a Python environment managed by conda, PyTorch, TorchTitan, vLLM and Monarch.

You also need Hugging Face Hub access to download models and datasets such as Qwen and GSM8K. A Weights & Biases account is optional and only needed for the monitoring dashboard the GRPO workflow mentions.

• 2+ NVIDIA GPUs for SFT, 3+ for GRPO (or ROCm hardware for the AMD path)
• Conda for the Python environment
• PyTorch (nightly, installed by the provided script), TorchTitan, vLLM and Monarch
• Hugging Face Hub access for models and datasets
• Optional: Weights & Biases account for monitoring

How to trigger the skill with natural requests

You do not need a special command. Describe the RL job you want in plain language, and the agent should load the skill when your request matches its description. The skill's description mentions clean RL abstractions, easy algorithm experimentation and scalable training with Monarch and TorchTitan, so using those ideas in your request helps.

Be specific about your model, dataset and GPU count. The skill's workflows depend on those details.

• "Set up a GRPO run with torchforge on Qwen2.5-7B-Instruct using GSM8K. I have three GPUs."
• "Write a custom loss class for torchforge that adds a clipped policy gradient and a KL penalty."
• "Help me scale my torchforge GRPO config from one node to two nodes with SLURM."
• "My torchforge run fails with an out-of-memory error during generation. What should I change?"

Example workflows the agent can follow

The skill includes three workflows. The GRPO workflow covers a math-reasoning run: a YAML config, a reward function, a launch command and a monitoring checklist. It assigns GPUs to the trainer, reference model and generator, and it suggests checking that entropy decreases and that KL divergence stays bounded.

The custom loss workflow shows a loss class that takes log probabilities, reference log probabilities, advantages and a padding mask. The distributed workflow shows how to set tensor and data parallel degrees and how to launch on SLURM or locally across multiple GPUs.

Safety notes: scripts and network access

This skill runs scripts and uses the network. Its install step runs a provided shell script, and its workflows download models and datasets from the Hugging Face Hub. Before you run anything, read the script and the commands the agent proposes, and confirm you are happy for them to run on your machine.

Multi-GPU jobs can use significant compute. Check your cluster or cloud budget before launching a long run, especially for the larger models the skill mentions.

• Read install.sh and any launch command before running it
• Check which models and datasets will be downloaded
• Use a conda environment so the install does not affect your other Python projects
• Watch GPU usage on shared or paid machines

Tips for getting good results

The skill's own documentation says torchforge is experimental and its APIs may change. Pin your environment once it works, and expect to adjust code when you update the library. Use the skill's two reference files when something breaks: troubleshooting.md covers GPU resource and memory issues, and api-reference.md covers the library's classes and batch format.

If you hit an out-of-memory error during generation, the troubleshooting guide suggests reducing the number of samples per prompt or the sequence length. If you see an insufficient GPU error, it suggests fewer processes per service, sharing a GPU between services, or running the reference model on CPU.

• Start with the smallest config that fits your GPUs, then scale up
• Ask the agent to explain each config field before you change it
• Keep a record of the config and library versions you used for each run