Distributed Training Guide
Guides an agent through multi-GPU PyTorch DDP training best practices.
- Repository
- aiming-lab/AutoResearchClaw
- Stars
- 14,802
- Licence
- MIT
- Files
- 1 file
What it does
When installed, this gives an AI agent a short checklist of best practices for training machine learning models across several GPUs at once using PyTorch's DistributedDataParallel. The agent will prefer DDP over the older DataParallel, set up the process group with NCCL, shard data with DistributedSampler, synchronize batch norm, save checkpoints only from the main process, scale the learning rate with the number of GPUs, and use gradient accumulation for larger effective batches. It is guidance only; it does not run anything by itself.
Great for
- ML engineers scaling a PyTorch training job from one GPU to several
- Researchers who want an agent to write or review multi-GPU training code
- Developers who are new to DDP and want a reminder of the standard steps
Skip it if
- Single-GPU or CPU-only projects, where these patterns add nothing (the guidance is specific to multi-GPU setups)
- Non-PyTorch frameworks such as TensorFlow or JAX, since the content only covers PyTorch DDP
- Production-critical decisions without expert review: the checklist is brief and does not cover edge cases like linear learning-rate scaling limits or cluster configuration
Try asking
- “Convert my single-GPU PyTorch training script to use DistributedDataParallel on 4 GPUs”
- “Review my training loop for multi-GPU issues like checkpoint saving and data sharding”
- “Explain how to scale my learning rate when I increase the number of GPUs”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 1 original files, the MIT licence and an attribution note.
distributed-training.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip distributed-training.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip distributed-training.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Give the agent your actual training script so it can apply the checklist to concrete code rather than giving generic advice.
- Verify any generated launch commands and learning-rate changes on a small run before a long multi-GPU job, since the skill does not test anything.
- Combine with the agent's own knowledge of your cluster setup, because the skill itself does not cover launchers like torchrun or multi-node networking.
What's inside
- SKILL.md
Redistributed unmodified from aiming-lab/AutoResearchClaw under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.