Guide · Agent skill
How to use the Caveman Experiment Manager agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The caveman-manage skill teaches an AI agent to inspect experiments in Caveman Cloud, a hosted service, by reading their state and results. It is written for operators and teams who want an assistant to summarize whether an experiment has enough evidence, without letting it change anything. After this guide you will know how to install the skill, how to ask for a review, and why the agent reports change requests as blocked.
Caveman Experiment ManagerJuliusBrussee/caveman · Apache-2.0 Download the skill Quick start
- 1Download and unzip the skill. Download the ZIP from this page and unzip it so the caveman-manage folder contains SKILL.md.
- 2Place it in the skills folder. Move the folder to ~/.claude/skills/ for personal use or to .claude/skills/ inside a project. Alternatively, upload it under Settings → Capabilities → Skills in the Claude apps.
- 3Confirm your Caveman access. Make sure you are logged in to Caveman Cloud with an identity that has access to the project, and that the Caveman MCP server or CLI is available.
- 4Ask for a read-only review. Name an experiment id, or ask the agent to list experiments, and request a review or explanation of the evidence.
- 5Check the evidence and proposed action. Read the state, sample sizes, guardrails and the single proposed action. Note any evidence incomplete flags.
- 6Treat change requests as blocked. If you ask to start, approve, cancel or roll back, expect a blocked response. Make any change through your own controlled process, then ask the agent to re-read the experiment.
What the skill makes the agent do
The agent reads the current state of a Caveman experiment and its results, then summarizes the evidence. That includes sample sizes for control and candidate, the quality or eval result, guardrail metrics such as latency, errors, cost, retries, drops and escalations when present, and whether the result is pending, failed, promotable or active.
The agent then proposes one next action. According to the skill file, it is not allowed to actually start, approve, cancel, promote or roll back an experiment. Requests to change experiments are reported as blocked until the server enforces the required safety gates.
- Reads experiment state and results only
- Summarizes evidence and flags anything missing as evidence incomplete
- Proposes one action with the experiment id and a reason
- Reports change requests as blocked; it does not execute them
Install the skill from the ZIP or the Claude apps
Download the ZIP from this page and unzip it. For a personal setup, place the skill folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.
If you use the Claude apps, you can upload the skill instead. Open Settings, go to Capabilities, then Skills, and upload the skill there.
The package contains a single file, SKILL.md.
- Personal: ~/.claude/skills/
- Project only: .claude/skills/ inside the project folder
- Claude apps: Settings → Capabilities → Skills
Connect the agent to Caveman Cloud
The skill needs one of two access paths. The preferred path is the Caveman MCP server, which exposes the caveman_context and caveman_experiment_get tools. The fallback is the Caveman CLI, which provides the caveman cloud commands.
You also need a logged-in Caveman Cloud identity with access to the project. Without that, the skill stops, because it cannot read anything.
- Caveman MCP server (caveman_context and caveman_experiment_get tools), or
- Caveman CLI (caveman cloud commands)
- A logged-in Caveman Cloud identity with access to the project
caveman cloud experiments list
caveman cloud experiments show <id>
caveman cloud experiments results <id>Trigger the skill with natural requests
You do not need special syntax. Ask in plain language about reviewing, inspecting, explaining or recommending on an experiment. The skill treats those requests as read-only. If you name an experiment id, the agent reads that experiment. If you do not, it lists experiments first.
Requests to start, approve, cancel, promote or roll back an experiment still trigger the skill, but the agent will explain that it cannot execute them and will report the request as blocked.
- "Review experiment 7f... and tell me whether it has enough evidence."
- "Explain the guardrail results for the current Caveman experiments."
- "List my Caveman experiments and recommend which one needs attention."
- "Approve experiment 7f..." (the agent will report this as blocked)
Read the output: evidence, recommendation and block
A typical review lists the lifecycle state, sample sizes, eval result, guardrails and a single proposed action. If a required field is missing, the agent says evidence incomplete and does not propose approval. It also does not treat missing data as a pass.
The skill shows its recommendation in a fixed format, with the proposed action, the reason and the execution status. Execution is marked as blocked until the server enforces its lifecycle and evidence gates.
- Absence of a field is never counted as a pass
- Unknown states and server errors are reported with the exact cave_snake_code
- Experiment lift is not treated as verified savings
Proposed action: approve experiment 7f...
Reason: candidate passed quality and every configured guardrail.
Execution: blocked until server-authoritative lifecycle and evidence gates ship.Safety notes: scripts, network and scope
The skill runs no scripts of its own. It does use the network, because it reads from Caveman Cloud through the MCP server or CLI. Access is limited by your logged-in identity and the server's role-based access control. The agent is told never to supply an organization id; project and tenant scope come from your login.
Treat the output as a review aid. The skill file forbids using experiment lift as proof of savings. Verified savings are recorded only by the signed ledger, so do not cite the agent's summary as evidence of savings.
- Runs scripts: no
- Uses network: yes, to read Caveman Cloud data
- Scope comes from your login and server RBAC, not from the agent
- Do not use output as proof of savings
What people use it for
- Checking whether a running experiment has enough control and candidate samples before anyone discusses it
- Reviewing guardrail metrics such as latency, errors and cost for a team's experiments
- Listing experiments in a project and identifying which one needs attention
- Auditing how an AI agent handles production control requests, since the skill shows it refusing to execute them
- Re-reading an experiment after an operator has acted, to confirm the server-observed state
Frequently asked questions
What is the caveman-manage skill?
It is an agent skill that lets an AI agent inspect experiments in Caveman Cloud by reading their state and results. It summarizes the evidence and proposes one next action, but it does not execute lifecycle changes.
Can the caveman-manage skill approve or cancel experiments?
No. The skill file says the agent is not allowed to start, approve, cancel, promote or roll back an experiment, and it reports such requests as blocked until the server enforces safety gates.
What do I need to use it?
You need either the Caveman CLI with caveman cloud commands or the Caveman MCP server with the caveman_context and caveman_experiment_get tools. You also need a logged-in Caveman Cloud identity with access to the project.
Is the caveman-manage skill free and what is its licence?
The skill is published under the Apache-2.0 licence. This page does not state pricing, and Caveman Cloud itself is a hosted service that requires an account.
How do I install the caveman-manage skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or into .claude/skills/ in a project. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.
Why does the agent say my experiment is evidence incomplete?
The skill treats absence as not a pass. If a required field such as a sample size, quality result or guardrail is missing, the agent reports evidence incomplete and does not propose approval.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use caveman-manage if you want an AI assistant to read Caveman Cloud experiment data and summarize whether the evidence is sufficient, or if you are auditing how agents handle production control requests. Do not use it if you expect the agent to start, approve, cancel or roll back experiments, if you have no Caveman Cloud account, or if you need proof of savings.
Download the skill