Guide · Agent skill
How to use the Cosmos Policy Evaluation agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The evaluating-cosmos-policy skill gives an AI agent step-by-step instructions for benchmarking NVIDIA Cosmos Policy, a robot-control model, inside two simulators: LIBERO and RoboCasa. It is for robotics researchers, ML engineers and teams who want to reproduce results or profile inference on local or cluster GPUs. After this guide you will know how to install the skill, ask for the work in plain language, and understand what the agent will run and what it needs.
Cosmos Policy EvaluationOrchestra-Research/AI-Research-SKILLs · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into .claude/skills/ inside your project.
- 2Set up the environment. Clone the cosmos-policy repository, build the Docker container from SETUP.md, and run uv sync with the libero group inside the container.
- 3Configure headless rendering. Export CUDA_VISIBLE_DEVICES, MUJOCO_EGL_DEVICE_ID, MUJOCO_GL and PYOPENGL_PLATFORM so simulations render without a screen.
- 4Run the smoke evaluation. Run one LIBERO trial on libero_10 with the public run_libero_eval module. Expect about 5 to 10 minutes on an A40 or A100.
- 5Check the results. Load the newest JSON file from the LIBERO log directory and inspect the output before continuing.
- 6Scale up if the smoke run passes. Run the four LIBERO suites with 50 trials each, or move to the RoboCasa workflow for a single task first.
What an agent skill is and what this one makes the agent do
An agent skill is a folder of instructions, reference files and sometimes scripts that an AI agent loads when a task matches the skill's description. This one is named evaluating-cosmos-policy and is built around the SKILL.md file plus two command reference files, one for LIBERO and one for RoboCasa.
With the skill loaded, the agent can set up a blank GPU machine for Cosmos Policy evaluation, configure headless rendering so simulations run without a screen, run a quick smoke test, parse the result logs, and scale up to full benchmark runs. Without it, the agent would have to work out these commands and flags from scratch.
- Set up the environment for Cosmos Policy LIBERO or RoboCasa evaluation
- Configure EGL headless rendering on a GPU machine
- Run a one-trial smoke evaluation before committing to a long run
- Read the JSON results from the official log directory
- Scale to full benchmark runs across LIBERO task suites or RoboCasa tasks
- Profile inference latency on local or cluster GPUs
How to install the skill
Option 1 is a personal install. Download the ZIP of the skill from this page, unzip it, and place the folder in ~/.claude/skills/ so it is available across your projects.
Option 2 is a project install. Unzip the skill into the .claude/skills/ folder inside the project where you will work, so it applies only there.
Option 3 is for the Claude apps. Download the ZIP, then upload it under Settings, then Capabilities, then Skills.
After installing, start a new session and check that the agent can see the skill before you rely on it.
- Personal: ~/.claude/skills/
- Project: .claude/skills/ inside your project folder
- Claude apps: Settings, then Capabilities, then Skills, then upload the ZIP
How to trigger the skill with natural requests
You do not need a special command. Agents choose skills by matching your request to the skill's description, which covers three situations: setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, and profiling inference latency on cluster or local GPU machines.
Be specific about the simulator, the goal and the machine. The more of these you state, the less the agent has to guess.
- Set up cosmos-policy on a fresh GPU machine and run a LIBERO smoke test
- Run a headless RoboCasa evaluation on one task with EGL rendering on our Slurm cluster
- Profile Cosmos Policy inference latency on one local GPU
- Parse the latest LIBERO eval logs and summarise the success rates
What you need before you start
The skill assumes a capable NVIDIA GPU. The source suggests an A40 or A100 with roughly 16 to 18 GB of VRAM. You also need Python 3.10 with the uv package manager, Docker (the setup says to build the supported container from SETUP.md), git access to GitHub, and Hugging Face access to download the nvidia/Cosmos-Policy-LIBERO-Predict2-2B checkpoint and related files.
The Python environment uses torch, mujoco, robosuite, transformers, and the cosmos-policy and robocasa packages. The skill's own setup steps install these through the repository, so you do not need to pick versions yourself.
- NVIDIA GPU, A40 or A100 suggested, about 16-18 GB VRAM
- Python 3.10 and uv
- Docker, for the container described in SETUP.md
- Git access to the cosmos-policy and robocasa-cosmos-policy repositories
- Hugging Face access for the checkpoints
git clone https://github.com/NVlabs/cosmos-policy.git
cd cosmos-policy
# Follow SETUP.md to build and enter the supported Docker container.
# Then, inside the container:
uv sync --extra cu128 --group libero --python 3.10Run a headless smoke test, then scale up
Start with headless rendering. The skill sets four environment variables so MuJoCo renders through EGL without a display. Then run a one-trial LIBERO smoke evaluation on the libero_10 suite. The skill's own time estimate for this is 5 to 10 minutes on one A40 or A100.
If the smoke run passes, the skill moves to full runs: all four LIBERO suites (libero_spatial, libero_object, libero_goal, libero_10) at 50 trials each. The skill estimates 2 to 4 hours per full LIBERO evaluation. RoboCasa has its own workflow and command file, with a single-task run of 2 trials taking about 10 to 15 minutes and all tasks taking 4 to 8 hours, per the skill's estimates.
- LIBERO smoke, 1 trial: 5-10 min, ~16 GB VRAM
- LIBERO full, 50 trials: 2-4 hours, ~16 GB VRAM
- RoboCasa single task, 2 trials: 10-15 min, ~18 GB VRAM
- RoboCasa all tasks: 4-8 hours, ~18 GB VRAM
export CUDA_VISIBLE_DEVICES=0
export MUJOCO_EGL_DEVICE_ID=0
export MUJOCO_GL=egl
export PYOPENGL_PLATFORM=eglSafety notes: scripts and network access
This skill runs scripts and uses the network. The setup clones repositories from GitHub, installs packages, and downloads model checkpoints and dataset files from Hugging Face. The evaluation commands run Python modules on your GPU and write logs to a local directory.
Read SKILL.md and the reference files before installing, and run the setup on a machine where those actions are acceptable. The skill covers simulation only. It does not target physical robots, so do not point it at hardware.
- Runs scripts: yes, Python evaluation modules and setup commands
- Uses network: yes, GitHub clones and Hugging Face downloads
- Writes logs to the local log directory given in the commands
- Simulation only; real-robot testing is out of scope
Tips for getting reliable results
Always run the smoke test first. It catches environment and checkpoint problems in minutes instead of hours. Use an interactive GPU shell when you are on a cluster, since the reference file recommends acquiring one GPU and running evaluations directly. Give each run a distinct run_id_note so the logs are easy to tell apart. The skill's commands fix the seed and set deterministic mode, which helps when you compare runs.
- Smoke test before any full run
- Use an interactive GPU shell on clusters (the reference uses srun)
- Label runs with run_id_note
- Keep the seed and deterministic flags unchanged when comparing results
What people use it for
- Reproducing Cosmos Policy results on the LIBERO benchmark suites
- Running RoboCasa single-task evaluations before a full sweep
- Setting up headless GPU evaluation on a Slurm cluster with an interactive shell
- Profiling inference latency of Cosmos Policy on one local or cluster GPU
Frequently asked questions
How do I install the evaluating-cosmos-policy skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into .claude/skills/ in a project folder. In the Claude apps, upload the ZIP under Settings, then Capabilities, then Skills.
What hardware do I need to run Cosmos Policy evaluations?
The skill suggests an NVIDIA A40 or A100 with about 16 to 18 GB of VRAM. Smoke runs take 5 to 10 minutes on LIBERO, while full benchmarks take hours.
Is the skill free to use?
The skill is listed under the MIT licence. Cosmos Policy itself is a separate NVIDIA project, and you still need access to its repositories and to Hugging Face for the checkpoints.
Can I use this skill to train or fine-tune Cosmos Policy?
No. The skill says training or fine-tuning from scratch should use the official Cosmos training docs. It is focused on evaluation, setup and inference profiling.
Does this work on real robots?
No. The skill covers simulation only, using LIBERO and RoboCasa. Real-robot evaluation is outside its scope.
What should I do if the smoke test fails?
Check the setup first: the GPU, the four headless rendering variables, the Docker container from SETUP.md, and Hugging Face access for the checkpoint files. Then rerun the one-trial smoke test before starting a full benchmark.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are a robotics researcher or ML engineer who needs to reproduce Cosmos Policy results on LIBERO or RoboCasa, or to set up headless GPU evaluation on a cluster. Pick something else if you need to train a model, test on physical robots, or do not have a suitable NVIDIA GPU and time for multi-hour runs. For other VLA policies, the skill points to fine-tuning-openvla-oft for OpenVLA and fine-tuning-serving-openpi for pi0.
Download the skill