Guide · Agent skill
How to use the Stat Result Validator agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The Stat Result Validator is an agent skill that gives an AI agent a checklist for reviewing a statistical research project before its conclusions are trusted. It is built for the output of the AutoResearchClaw research pipeline, which expects a specific folder layout. This guide explains what the skill makes the agent check, how to install it, how to trigger it, and where its limits are.
Stat Result Validatoraiming-lab/AutoResearchClaw · MIT Download the skill Quick start
- 1Download the ZIP. Download the skill ZIP from the item page and unzip it. Confirm the folder contains SKILL.md.
- 2Choose an install location. Place the skill folder in ~/.claude/skills/ for personal use, or in a project's .claude/skills/ folder to limit it to that project.
- 3Or upload in the Claude apps. In the Claude apps, go to Settings → Capabilities → Skills and upload the skill instead.
- 4Prepare the research folder. Make sure progress/<TOPIC_ID>/ and experiments/<TOPIC_ID>/ contain the required files listed in the skill.
- 5Ask in plain language. Request a validation using words such as audit, quality gate, or claims, and include the topic ID.
- 6Review PASS, WARN and FAIL verdicts. Read each claim's verdict and the list of missing pieces, then fix gaps and re-run.
What is an agent skill and what does this one do?
An agent skill is a folder containing a SKILL.md file with instructions that an AI agent loads when a task matches the skill's description. This skill tells the agent how to audit a statistical research chain: the problem formulation, the method, the theory, the experiments, the comparisons, and the final claims.
The agent checks that the problem is clearly defined, the method fits the problem, theory is stated, experiments test the claims fairly, and the final claims are not stronger than the evidence. Each claim is then marked PASS, WARN, or FAIL, with missing pieces listed.
- PASS: formulation, theory, experiments, and comparisons support the claims.
- WARN: usable but has limitations that must be disclosed.
- FAIL: missing formulation, theory, evidence, or fair comparison prevents a valid conclusion.
What files and folders must exist first?
The skill assumes an existing research output folder that follows the progress/ and experiments/ layout. Many checks look for specific files, so projects with a different structure will need the agent to adapt the checklist.
Required for all topics are seven step files under progress/<TOPIC_ID>/, plus config.yaml, metrics.json, run_manifest.json, comparison_summary.md, claim_verdicts.json, report/paper.md and README.md under experiments/<TOPIC_ID>/. Analysis-specific source files and raw outputs should live under experiments/<TOPIC_ID>/src/ and experiments/<TOPIC_ID>/results/.
progress/<TOPIC_ID>/step0_problem_formulation.md
progress/<TOPIC_ID>/step1_method_proposal.md
progress/<TOPIC_ID>/step2_theory_analysis.md
progress/<TOPIC_ID>/step3_experimental_evaluation.md
progress/<TOPIC_ID>/step4_comparison.md
progress/<TOPIC_ID>/step5_result_synthesis.md
progress/<TOPIC_ID>/step6_quality_audit.md
experiments/<TOPIC_ID>/config.yaml
experiments/<TOPIC_ID>/results/metrics.json
experiments/<TOPIC_ID>/results/run_manifest.json
experiments/<TOPIC_ID>/results/comparison_summary.md
experiments/<TOPIC_ID>/results/claim_verdicts.json
experiments/<TOPIC_ID>/report/paper.md
experiments/<TOPIC_ID>/README.mdHow do you install the Stat Result Validator skill?
Download the ZIP from the item page and unzip it. The skill folder contains SKILL.md. Place the unzipped folder in one of these locations: ~/.claude/skills/ for personal use across projects, or a project's .claude/skills/ folder to limit it to one project.
In the Claude apps, you can instead upload the skill under Settings → Capabilities → Skills.
- Personal install: put the skill folder in ~/.claude/skills/
- Project install: put the skill folder in your project's .claude/skills/
- Claude apps: Settings → Capabilities → Skills, then upload
How do you trigger the skill with natural requests?
The skill is meant to run after formulation, method proposal, theory, experimental evaluation, comparison, and result synthesis. You do not need special syntax. Ask in plain language and include the topic ID so the agent knows which folders to read.
The skill's metadata lists trigger keywords including validation, audit, formulation, theory, comparison, claims, statistical sanity, and quality gate. Using these words in your request helps the agent match the skill.
- Validate the results for topic 042 before I report them.
- Run a quality gate on the claims in progress/042 and experiments/042.
- Audit the theory and comparison sections of this statistical analysis.
- Check whether the final claims are supported by the experiments and theory.
What should you expect the agent to produce?
The agent works through the checklist stages: formulation, method, theory, experimental evidence, comparison, and final claims. Some issues are blocking failures. Examples include no target or estimand, claims that cannot be measured, no theory section at all, and final claims stronger than the theory supports.
Each final claim is traced through formulation, method, theory, experiment, and comparison. Missing pieces are listed so you can fix them before reporting.
- No target or estimand is a blocking formulation failure.
- A missing theory section is a blocking theory failure.
- Runtime reductions should be recorded in run_manifest.json.
- Claim verdicts should cite theoretical and empirical support when available.
What are the safety notes and limits?
The skill runs no scripts and uses no network access. The skill file does not specify additional software, libraries, API keys or accounts.
The skill is a checklist applied by an AI and can miss subtle statistical errors. It is not a replacement for a qualified statistician's review on high-stakes decisions. It is also not suited to non-statistical content, since the checks are written for statistical research chains.
- Runs scripts: no
- Uses network: no
- Requires: an existing progress/ and experiments/ research folder
- Not a substitute for a qualified statistician on high-stakes decisions
Tips for getting useful results
Give the agent a complete folder before asking for validation, because missing files will appear as gaps in the report. If a check fails, fix the underlying file and rerun the skill rather than editing the verdict.
If your project uses a different layout, tell the agent which files correspond to each required item so it can adapt the checklist, and note any adaptation in your report.
- Confirm all required files exist before running the audit.
- Re-run after fixes so verdicts reflect the current files.
- Treat WARN items as limitations to disclose, not as silent passes.
What people use it for
- Checking a statistical analysis for a clear target, stated assumptions and evaluation criteria before sharing it.
- Reviewing a paper draft to see whether its claims are supported by its experiments and theory.
- Running a quality gate in an AutoResearchClaw pipeline before results are reported.
- Preparing a structured second opinion for a student's thesis chapter that follows the required layout.
Frequently asked questions
Is the Stat Result Validator skill free?
The source file is published under the MIT licence, as listed on the item page. The skill runs no scripts and needs no API keys or accounts that the skill file specifies.
What do I need to use the stat-result-validator skill?
You need an existing research output folder that follows the progress/ and experiments/ layout described in the skill. The skill file does not specify other software, libraries or accounts.
Where do I install an agent skill like this one?
Put the unzipped skill folder in ~/.claude/skills/ for personal use or in a project's .claude/skills/ folder. In the Claude apps, you can upload it under Settings → Capabilities → Skills.
Why does the validator say a file is missing?
Many checks look for specific files such as step files, config.yaml, metrics.json and claim_verdicts.json. If your project uses a different structure, the checks may fail, so you will need to adapt the checklist or rename files to match.
Can I use this skill for non-statistical research?
No. The checks are written for statistical research chains, so non-statistical content is not a good fit.
Can this replace a statistician's review?
No. It is a checklist applied by an AI and can miss subtle statistical errors, so high-stakes decisions still need a qualified statistician.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are a researcher, student or team running the AutoResearchClaw pipeline and want a structured quality gate or second opinion on statistical claims. Pick something else if your project does not follow the progress/ and experiments/ layout, if your content is non-statistical, or if you need a full review for a high-stakes decision.
Download the skill