Training Health Monitor
Watches ML training runs over time and flags NaN, divergence, or plateaus, deciding whether to continue or stop.
- Stars
- 18,482
- Licence
- MIT
- Files
- 1 file
What it does
Once installed, the agent keeps an interactive terminal session open and checks a machine-learning training run roughly every 30 minutes. It reads metrics from Weights & Biases (WandB) or fallback log files, looks for problems such as NaN values, loss spikes, divergence, or flat plateaus, and writes a short report with a CONTINUE, WAIT, or STOP decision. If it decides a run is broken, the skill tells the agent to stop the training task and save evidence and notes.
Great for
- ML engineers running long training jobs who want an agent to watch loss and eval curves
- Researchers who want a consistent written record of run health and stop decisions
- Teams using WandB who want quality-level checks beyond basic process monitoring
Skip it if
- Checking whether a process, tmux session, or GPU is alive; the skill says watchdog-style tools handle that better
- Automatic stopping of costly jobs without human review, since the agent may halt training on its own judgement from partial logs
- Runs whose logs or metrics are not reachable, because the skill can only report connectivity issues and wait
Try asking
- “Monitor my WandB run at this URL and tell me every 30 minutes if the loss is diverging”
- “Check my fallback training log over SSH for NaN values and recommend continue or stop”
- “Watch this fine-tuning run against an eval baseline of 0.82 and write a final summary if you stop it”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 1 original files, the MIT licence and an attribution note.
training-check.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip training-check.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip training-check.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Provide a stop_command and the launch method up front so a STOP decision can actually halt the job safely.
- State your baseline and success metrics before the first check so the agent judges against them rather than guessing.
- Review the saved evidence and project notes after any STOP decision before relying on it.
What's inside
- SKILL.md
Redistributed unmodified from wanshuiyin/Auto-claude-code-research-in-sleep under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.