A/B Test Planner
Plans, designs and analyses A/B tests and experiments with sound statistics
- Repository
- sickn33/agentic-awesome-skills
- Stars
- 49,023
- Licence
- MIT
- Files
- 4 files
What it does
Once installed, the agent helps you plan controlled experiments: it asks what you want to improve, writes a clear testable hypothesis, picks primary, secondary and guardrail metrics, and recommends sample sizes and test duration. It also gives a pre-launch checklist and warns about common mistakes such as stopping a test early or misreading p-values. The skill is mostly written guidance and reference tables rather than code.
Great for
- Marketers planning landing page, pricing or signup experiments
- Product and growth teams setting up an experimentation programme
- Developers implementing client-side or server-side variant assignment
Skip it if
- Teams with very little traffic, where any test will be underpowered and the skill cannot fix that
- Situations needing a formal statistical analysis of existing data; the skill gives guidance, not a verified calculation pipeline
Try asking
- “Help me design an A/B test for our pricing page and pick the primary metric”
- “Calculate the sample size for a signup button test with a 5% baseline conversion”
- “Write a hypothesis for testing a shorter checkout form”
- “Review my test results and tell me whether I should stop the experiment yet”
Install in 3 steps
- 1
Download the skill
The ZIP contains the 4 original files, the MIT licence and an attribution note.
ab-testing.zip - 2
Add it to your agent
Claude Code (all projects)
mkdir -p ~/.claude/skills && unzip ab-testing.zip -d ~/.claude/skills/Claude Code / Agent SDK (this project only)
mkdir -p .claude/skills && unzip ab-testing.zip -d .claude/skills/Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.
- 3
Ask for the task
No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.
Get better results
- Give the agent your baseline conversion rate, weekly traffic and the smallest effect you care about so the sample size estimate is meaningful
- Use the external calculators it links (Evan Miller, Optimizely) to check numbers before committing to a test length
- If you have a product marketing context file, the skill will read it first, so keep it up to date
What's inside
- evals/evals.json
- references/sample-size-guide.md
- references/test-templates.md
- SKILL.md
Redistributed unmodified from sickn33/agentic-awesome-skills under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.