Guide · Agent skill
How to use the Result-to-Claim Gate agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The result-to-claim skill is an agent skill for researchers who have finished their main experiments and need to know what those results actually support. It makes the agent collect results, check that the numbers it cites exist in the result files, get an outside reviewer's judgment, and recommend whether to pivot, run supplementary experiments, or confirm the claim. This guide explains what agent skills are, how to install this one, how to trigger it, and what it needs before you run it.
Result-to-Claim Gatewanshuiyin/Auto-claude-code-research-in-sleep · MIT Download the skill Quick start
- 1Download the ZIP. Download the result-to-claim skill ZIP from this directory's skill page.
- 2Install the skill. Unzip it into ~/.claude/skills/ for all projects, or into .claude/skills/ in one project. In the Claude apps, upload it under Settings, then Capabilities, then Skills.
- 3Write down your claim. Record the intended claim and the experiments that test it in your research contract file so the agent can read it.
- 4Ask for a check after experiments finish. Make a natural request such as asking whether the results support your claim, and point the agent to your W&B runs or log files.
- 5Let the pre-check run. The agent builds .aris/claims.json and runs the evidence check. Any number not found in its source file is flagged before the reviewer is called.
- 6Read the verdict. Review the reviewer's judgment and the recommended action: pivot, supplement, or confirm.
What the result-to-claim skill makes the agent do
An agent skill is a folder of instructions that an AI agent loads when a request matches the skill's purpose. This one is designed for the stage after experiments finish and before you write a paper or start ablations. Its job is to turn raw numbers into a judgment about the claim they were meant to test.
The skill gathers results from sources such as Weights & Biases runs, experiment log files, and server logs. It then assesses what the numbers support, what they do not support, and what evidence is still missing. Before asking an outside reviewer model for a judgment, it runs a mechanical check that cited numbers really appear in the result files, which catches invented evidence. It ends by recommending one of three actions: pivot, run supplementary experiments, or confirm the claim.
Install the result-to-claim skill
Download the skill ZIP from the skill's page on this directory. Then choose one of the following options.
For a personal install across all your projects, unzip the file into ~/.claude/skills/. For a single project, unzip it into the .claude/skills/ folder inside that project. In the Claude apps, you can instead upload the ZIP under Settings, then Capabilities, then Skills.
- Personal install: unzip into ~/.claude/skills/
- Project install: unzip into .claude/skills/ in your project
- Claude apps: Settings → Capabilities → Skills → upload the ZIP
Trigger the skill with a natural request
You do not need a special command. The skill's description tells the agent to use it after experiments finish, so a plain request about your results is usually enough. You can also pass a short description of the experiment or a Weights & Biases run as the argument, since the skill accepts an experiment description or run reference as context.
Make sure your intended claim is stated somewhere the agent can read, such as docs/research_contract.md or idea-stage/docs/research_contract.md. The verdict depends on that claim.
- Our experiments are done. Do the results support the claim that the new loss improves accuracy on the held-out set?
- Check whether these W&B runs back up the claim in our rebuttal draft.
- Before I start the ablations, judge what the main results actually show.
Run the evidence pre-check
Before the outside reviewer is called, the skill builds a list of claims with their cited values and source files, saves it to .aris/claims.json, and runs a helper script called evidence_check.py. The script checks that each value appears in the cited file. Claims whose numbers are not found are marked as evidence not found and are not sent for debate.
The pre-check only confirms that the evidence exists. Whether that evidence supports the claim is still decided by the reviewer in the next step. If the helper script cannot be found, the skill prints a warning and continues without the pre-check.
- Each claim needs an id, a value, and a source path relative to the project root.
- Statuses are verified, value_not_found, path_missing, or unparseable.
python3 tools/evidence_check.py . --batch .aris/claims.jsonGet the reviewer judgment and act on the verdict
The skill sends the intended claim, the experiments, the key numbers, and the pre-check results to an outside reviewer model through configured MCP tools. Codex is the default route in the skill. The skill lists alternatives, and it can use a different reviewer if one is configured in .aris/reviewer.json.
The reviewer returns a verdict, and the skill routes to the next action: pivot if the claim is not supported, supplement if key evidence is missing, or confirm if the results hold up.
- Pivot: the results do not support the claim as stated
- Supplement: the claim may hold but evidence is missing
- Confirm: the results support the claim
Safety notes: scripts and network access
This skill runs scripts and uses the network. Its allowed tools include shell commands, file reading and writing, and MCP calls to reviewer models. The pre-check runs a Python 3 script in your project. Reviewer calls send your experiment summaries and numbers to the reviewer service you have configured.
If you use Weights & Biases or read logs over SSH, the skill will access those systems with the credentials you have set up. Review the reviewer and log access settings before running it on confidential or unpublished results.
- Python 3 is required for the pre-check script; without it the pre-check is skipped with a warning.
- Reviewer calls require MCP tools configured for your chosen reviewer.
- Optional: Weights & Biases account and API access, and SSH access to a training server.
Tips for getting a useful verdict
Write the intended claim down before you look at the results. The verdict is only as meaningful as the claim it tests. Use the skill on main results and claims you plan to publish, not on every run.
Run it once after results change. The skill's own instructions say not to wrap it in a loop or a timer, because the verdict only changes when the results change.
- State the claim and the baseline comparison explicitly.
- Record known confounds in your experiment log so the reviewer sees them.
- Do not use it for routine sanity checks or debugging runs.
- Skip it if the project has no defined research claim.
What people use it for
- Deciding whether main results justify the central claim of a paper before writing it.
- Checking that every number cited in a rebuttal or review response appears in the actual result files.
- Getting a structured second opinion when a set of experiments gives mixed results.
- Identifying which evidence is still missing before you commit to additional ablation runs.
Frequently asked questions
Is the result-to-claim skill free to use?
The source material for this directory listing describes the skill as MIT licensed. Check the licence file in the download for the exact terms. Reviewer models and services you connect, such as Codex, may have their own costs and terms.
What do I need to run the result-to-claim skill?
You need an agent that supports skills and an outside reviewer model reachable through configured MCP tools. Python 3 is needed for the evidence pre-check script. Weights & Biases access and SSH access to a training server are optional, depending on where your results live.
Where do I put the result-to-claim skill folder?
Unzip the downloaded ZIP into ~/.claude/skills/ to use it everywhere, or into .claude/skills/ inside a project to use it only there. In the Claude apps, upload the ZIP under Settings, then Capabilities, then Skills.
What if evidence_check.py is missing?
The skill prints a warning and skips the deterministic pre-check, then still runs the reviewer judgment. The skill's instructions suggest rerunning the ARIS install script, exporting ARIS_REPO, or copying the helper into a tools/ folder.
Can I run it on a schedule to keep results up to date?
No. The skill's instructions say not to wrap it in a loop, scheduler, or cron job, because the verdict only changes when the results change. Run it once after your experiments finish or after results change.
How is this different from just asking an AI to review my results?
This skill adds a mechanical check that cited numbers exist in your result files before any judgment is made, and it routes the outcome to a specific next action. It is meant for main results and gating claims rather than every run.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you have finished main experiments, have a stated research claim, and want an independent check before writing a paper or rebuttal. Pick something else for routine debugging runs, scheduled re-checks, or projects that have no defined claim to test.
Download the skill