Guide · Agent skill
How to use the Monitor Experiment Results agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The monitor-experiment skill is a set of instructions that teaches an AI agent how to check on machine learning experiments that are running on remote machines and report back the results. It is for researchers who launch training jobs on SSH servers, Vast.ai GPU rentals or Modal, and want a quick status report without logging in by hand. After reading this guide you will know how to install the skill, how to ask for a status check in plain language, and what the agent will do on your behalf.
Monitor Experiment Resultswanshuiyin/Auto-claude-code-research-in-sleep · MIT Download the skill Quick start
- 1Download the ZIP. Download the monitor-experiment ZIP from this page. It contains the SKILL.md file.
- 2Unzip into a skills folder. Unzip it into ~/.claude/skills/ for a personal install, or into .claude/skills/ in your project. In the Claude apps you can instead upload it under Settings → Capabilities → Skills.
- 3Prepare your project. Create a CLAUDE.md file that describes your server, GPU and wandb settings. Add vast-instances.json if you use Vast.ai.
- 4Check your tools. Confirm you have SSH access, screen on the remote machine, and the vastai or Modal CLI if you use them.
- 5Ask for a status check. Say something like "check results" or "is it done", optionally with a server alias or screen name.
- 6Review the table. Read the raw numbers and the comparison table first, then the interpretation and any flagged problems.
- 7Handle cost reminders. If the agent reminds you to stop a finished Vast.ai instance, decide whether to run the destroy command it suggests.
What an agent skill is and what this one does
An agent skill is a folder containing a SKILL.md file that tells a Claude agent how to handle a particular kind of task. This skill covers one task: finding out where your experiments stand and collecting their output.
When you ask for a status check, the skill guides the agent through a fixed sequence. It lists the screen sessions on your server, captures the last lines of output from each one, looks for JSON result files, and, if you use Weights & Biases, pulls training metrics. It then presents the numbers in a comparison table with a short interpretation.
- Checks running jobs over SSH, on Vast.ai instances, or on Modal
- Reads screen session output and log files
- Picks up JSON result files and Weights & Biases metrics
- Shows raw numbers first, then flags obvious problems such as NaN, divergence or negative deltas
- Reminds you to stop paid Vast.ai instances when their work is finished
- Sends a notification if you have configured Feishu
Install the monitor-experiment skill
Download the ZIP for this skill from this page. It contains the SKILL.md file. Unzip it so that the folder sits in one of two places. For a personal setup, use ~/.claude/skills/. For a single project, put it in the project's .claude/skills/ folder.
If you use the Claude apps instead of the command line, open Settings, go to Capabilities, and upload the skill under Skills.
- Personal install: unzip into ~/.claude/skills/
- Project install: unzip into .claude/skills/ inside your project
- Claude apps: Settings → Capabilities → Skills, then upload the skill
Set up the requirements
The skill assumes you already have experiments running. It does not start jobs. What you need depends on where your jobs run.
For an SSH server you need SSH access and the screen tool installed on the remote machine. For Vast.ai you need the vastai CLI and a vast-instances.json file that lists each instance's host and port. For Modal you need the Modal CLI. For Weights & Biases you need a W&B account and Python with the wandb package on the remote server.
Create a CLAUDE.md file in your project that describes your server, GPU and wandb settings. The skill reads these settings to decide which checks to run. The optional ~/.claude/feishu.json file enables Feishu notifications.
- SSH server: ssh client and screen on the remote machine
- Vast.ai: vastai CLI and vast-instances.json
- Modal: Modal CLI
- Weights & Biases: wandb Python package on the server and a W&B account
- Project: CLAUDE.md describing server, GPU and wandb settings
- Optional: ~/.claude/feishu.json for Feishu notifications
Trigger the skill with natural requests
The skill activates when you use phrases like the ones in its description. You can also pass a server alias or screen name after the skill name to narrow the check.
Plain requests work well. The agent reads your words and decides whether this skill fits.
- "check results"
- "is it done"
- "monitor the baseline run on my GPU box"
- "show me the experiment output from the screen called train-a"
- "monitor my vast.ai instance and tell me the cost so far"
/monitor-experiment [server-alias or screen-name]What the agent checks and reports
For an SSH server the agent runs screen -ls, then captures the last 50 lines from each session. For Vast.ai it uses the stored host and port to run the same check, and also runs vastai show instances. For Modal it runs modal app list and modal app logs. Modal apps stop automatically when they finish, so a missing app usually means it has completed.
When results are ready, the agent builds a table with the experiment, metric, delta versus baseline and status. Before giving any interpretation, it shows the raw numbers. It notes whether runs are still going by checking progress bars and iteration counts.
- Screen output, JSON results and W&B run summaries are the sources
- Compares against the correct baseline with the same configuration
- If results look wrong, it checks training logs for errors before concluding
| Experiment | Metric | Delta vs Baseline | Status |
|-----------|--------|-------------------|--------|
| Baseline | X.XX | — | done |
| Method A | X.XX | +Y.Y | done |Safety notes: scripts and network access
This skill runs scripts and uses the network. It is allowed to run ssh and echo commands, and to read and write files. The agent will connect to your remote servers, run commands there, and read job output and W&B data. Only use it where that access is acceptable.
Do not use it on a server or with credentials that are shared or sensitive. The skill is not a judge of scientific quality. It flags only machine-checkable problems such as NaN values, divergence and negative deltas.
- Runs remote commands over SSH on your servers
- Reads job outputs and, if enabled, W&B data over the network
- Sends Feishu notifications only if ~/.claude/feishu.json is configured and not set to off
- Verify results yourself before drawing scientific conclusions
Tips for useful status reports
Keep your CLAUDE.md accurate, because the skill relies on it for server, GPU and wandb settings. Name your screen sessions clearly so the output is easy to match to experiments. For Vast.ai, keep vast-instances.json up to date with the correct host and port.
Ask for the check again after a run is done to get the final table. If you see a cost reminder for a Vast.ai instance, decide whether to destroy it with the command the agent gives you.
- Use clear screen names such as baseline or method-a
- Keep vast-instances.json current with host, port and $/hr
- Use the W&B dashboard link in the summary to inspect curves yourself
What people use it for
- Checking whether a training run on a remote GPU server has finished and reading its last log lines
- Comparing a new method's eval loss against a baseline in a single table
- Pulling W&B loss curves and learning rate history for a run that is still going
- Reviewing the running cost of a Vast.ai instance and getting a reminder to stop it when work is done
- Reading the latest JSON result file from a results folder after a batch of experiments
Frequently asked questions
Is the monitor-experiment skill free?
The skill is released under the MIT licence, as stated on this page. The services it works with, such as Vast.ai, Modal and Weights & Biases, may charge for their own use, so check their pricing separately.
What do I need to install the monitor-experiment skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder. In the Claude apps you can upload it under Settings → Capabilities → Skills.
What does the monitor-experiment skill require to run?
It needs SSH access to your experiment server with screen installed there. Vast.ai needs the vastai CLI and vast-instances.json, Modal needs the Modal CLI, and W&B needs the wandb package and an account.
Why does the agent say my experiment is done when it is still running?
The skill checks screen sessions, logs, JSON files and W&B state, so an unclear output can be misread. Ask it to check the progress bars and iteration counts, and look at the W&B run status or the log tail yourself.
Does the skill judge whether my research results are good?
No. It flags only machine-checkable problems such as NaN, divergence and negative deltas, and it is meant for status rather than quality. You still need to assess scientific validity yourself.
Are there alternatives for monitoring experiments?
Yes. You can check screen sessions and logs by hand, use the Weights & Biases dashboard directly, or use the Vast.ai and Modal consoles. This skill is useful if you want an agent to gather those checks and summarize them in one place.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you run training jobs on SSH servers, Vast.ai or Modal and want an agent to gather status, metrics and cost reminders in one table. Skip it if you have no remote experiments, or if the server and credentials are shared or sensitive, since the agent runs commands over SSH and reads job output.
Download the skill