Guide · Agent skill
How to use the Caveman Evidence Review agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
Caveman Evidence Review is an agent skill for people who use Caveman Cloud, a service that tracks LLM API spending and performance. An agent skill is a folder of instructions (a SKILL.md file) that an AI agent loads when your request matches it, so the agent follows a repeatable method. This guide explains what the skill does, how to install it, how to phrase requests so it triggers, and what to check before you rely on its answers.
Caveman Evidence ReviewJuliusBrussee/caveman · Apache-2.0 Download the skill Quick start
- 1Get the ZIP. Download the caveman-evidence-review ZIP from our page and unzip it.
- 2Place the folder. Move the skill folder into ~/.claude/skills/ for personal use, or into .claude/skills/ in a project. You can also upload it under Settings → Capabilities → Skills in the Claude apps.
- 3Install and log in to Caveman CLI. Install the Caveman CLI, then run caveman login.
- 4Select a project. Choose a Caveman Cloud project so the skill's reads are scoped correctly.
- 5Ask a plain question. Ask what Caveman found or where LLM spend goes, naming a workflow or time period if you have one.
- 6Check the evidence. Review the trace IDs and time windows in the report before acting on any finding.
What the skill makes the agent do
When you ask what Caveman found or where your LLM spend goes, the skill tells the agent to work from current Caveman data rather than guesses about your repository. It pulls cost reports, Cave Score, workflows, traces, latency, errors, routing and savings figures, then reports findings with trace IDs and exact time windows.
The skill is strictly read-only. It is told never to start, approve, cancel or roll back an experiment, and to leave prompt, completion, tool and artifact content alone unless you explicitly ask for payload review.
- Measured provider-complete list-price cost is kept separate from inferred daily headroom and verified ledger savings.
- Empty results are reported as no current signal, not as zero cost or zero risk.
- Conclusions about cause are only drawn from trace evidence and comparisons, not from a single aggregate.
Install the skill from the ZIP
Download the ZIP for this skill from our page and unzip it. Then place the skill folder in one of two locations: your personal skills folder at ~/.claude/skills/, or a project-specific .claude/skills/ folder inside the project you work in. A personal install is available in every project; a project install applies only to that project.
You can also upload the skill in the Claude apps under Settings → Capabilities → Skills.
- Personal: ~/.claude/skills/
- Per project: .claude/skills/ inside your project folder
- Claude apps: Settings → Capabilities → Skills, then upload
Set up Caveman before you start
The skill depends on the Caveman CLI. Install it, then log in and select a Caveman Cloud project. The skill will stop and ask you to run login or project selection if either is missing, rather than guessing a project.
The skill prefers Caveman MCP tools (caveman_context, caveman_report, caveman_plan, caveman_trace_search, caveman_trace_get). If those are not available, it falls back to CLI commands such as caveman cloud whoami and caveman cloud projects list.
caveman loginTrigger the skill with natural requests
You do not need a special command. Ask in plain language about what Caveman found or where your model spend goes. Name a workflow, model or time period when you can, because the agent scopes its reads to those details and to the project selected in Caveman.
Narrow questions get narrow checks. The skill is designed to read the shortest set of reports that can answer your question, so a focused question produces a faster and tighter review.
- What did Caveman find about our spend this week?
- Where is our LLM spend going by workflow?
- Why did cost spike on the summarisation workflow yesterday?
- Are we seeing retry loops or latency problems in traces from the last 24 hours?
- Compare our model mix for the checkout workflow against last month.
Read the report the agent produces
The skill asks for a fixed report shape: scope with project and time window, measured cost, verified savings, inferred headroom, numbered findings with aggregate evidence and trace IDs, unproven explanations, one next read-only check, and a possible action that is only a proposal.
Treat the sections as separate. Measured cost, verified savings and inferred headroom must not be added together or relabelled. A catalog subtotal is not an invoice, and an experiment result is not verified savings.
- Check the stated window and basis before reading any trend.
- Look at the trace IDs to verify each finding yourself.
- Treat items under Unproven as hypotheses until a control or eval confirms them.
Know the safety and data boundaries
The skill runs scripts and uses the network, because it queries Caveman Cloud through the CLI or MCP tools. It does not fetch raw prompt or completion content by default. Metadata, spans, timing, models, token counts, status and optimizer attribution are enough for the default review.
If a finding needs an action such as changing an experiment or routing, the skill defers to another skill, caveman-manage, which handles read-only lifecycle review and a safety gate. This skill itself will not make the change.
- Payload retrieval is off unless you explicitly request payload review.
- The skill never supplies an organisation ID; reads are scoped to the selected project.
- It only works against Caveman Cloud data.
What people use it for
- Finding which workflow drives most of your LLM cost over a chosen window.
- Comparing model mix across two time periods for one workflow.
- Investigating a cost spike or retry loop by pulling representative traces.
- Checking whether verified savings and inferred headroom are being reported separately and correctly.
- Getting a read-only summary of Caveman findings to share with an engineering or finance team.
Frequently asked questions
Is the Caveman Evidence Review skill free?
The source does not state pricing or licence terms beyond the licence listed for the skill, which is Apache-2.0. The skill itself only works with Caveman Cloud data, so you will need access to that service.
What do I need to use this skill?
You need the Caveman CLI installed and logged in with caveman login, a selected Caveman Cloud project, and either the Caveman MCP tools or the CLI fallback commands. The skill also runs scripts and uses the network.
How do I install caveman-evidence-review?
Download the ZIP from our page and unzip it into ~/.claude/skills/ for personal use or into .claude/skills/ in a project. In the Claude apps you can instead upload it under Settings → Capabilities → Skills.
Can this skill change my experiments or routing?
No. The skill is read-only and must not start, approve, cancel or roll back experiments. For lifecycle review and a safety gate, the source points to the separate caveman-manage skill.
Does the skill read my prompts and completions?
Not by default. Prompt, completion, tool and artifact payloads are only fetched if you explicitly ask for payload review. Metadata, spans, timing, models, token counts and status are used otherwise.
What should I do if the skill says login or project selection is missing?
Run caveman login and select a Caveman Cloud project, then ask again. The skill is designed to stop rather than guess a project.
How does this compare with just asking an AI about my LLM costs?
The skill is built to ground conclusions in current Caveman data with trace IDs and time windows, and to avoid claiming causes from aggregates alone. A general chat answer without that data is more likely to be a guess.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
This skill suits engineering teams and operators who already use Caveman Cloud and want evidence-backed, trace-cited answers about LLM spend, cost spikes and latency. It is a poor fit if you do not use Caveman Cloud, or if you need the agent to change experiments or routing, since it is deliberately read-only and defers actions to caveman-manage.
Download the skill