Skip to content

Experiment Queue for GPU Batches

Runs large batches of ML experiments on remote GPU servers with retries and state tracking.

Stars
18,482
Licence
MIT
Files
3 files

What it does

When installed, the agent can take a large set of machine learning training jobs (for example, a multi-seed or multi-config grid of 10 or more) and manage them on a remote SSH GPU server. It starts jobs on free GPUs, retries jobs that run out of GPU memory after a delay, cleans up leftover terminal sessions, and launches the next group of jobs only after the previous group has finished. It also keeps a state file so progress can be checked overnight.

Great for

  • ML researchers running multi-seed sweeps or hyperparameter grids on a shared SSH GPU server
  • Teams that need teacher-then-student training chains to start automatically once prerequisites exist
  • People who already use single-experiment tooling and now need batch orchestration for 10+ jobs

Skip it if

  • Single ad-hoc experiments, where the simpler run-experiment workflow is enough
  • Modal or Vast.ai deployments, which the skill says have their own orchestration
  • Experiments needing manual inspection between runs, since failed jobs are parked as stuck for a human to review

Try asking

  • “Run a grid of 21 seeds across 12 configurations on my GPU server and retry any out-of-memory failures”
  • “Set up a teacher-then-student chain: train the teacher models, then auto-start distillation once checkpoints exist”
  • “Check how many of my overnight experiment jobs are done, running, or pending”
Step-by-step guide: how to use Experiment Queue for GPU BatchesInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    experiment-queue.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip experiment-queue.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip experiment-queue.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Set PROJECT_DIR before running and review the generated manifest.json before launching, since the scheduler will run commands on your server.
  • Do not wrap the scheduler in an extra polling loop; use its status output to check progress, as the skill itself warns about race conditions.
  • Use gpu_free_threshold_mib and oom_retry settings to match a shared server's memory behavior.

What's inside

  • scripts/build_manifest.py
  • scripts/queue_manager.py
  • SKILL.md

Redistributed unmodified from wanshuiyin/Auto-claude-code-research-in-sleep under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.