When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate t
Install
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add alirezarezvani/claude-skills --skill ab-test-setup -a cursor -y
When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
Questions
How do I install ab-test-setup?
Download .gemini/skills/ab-test-setup/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
Where does ab-test-setup live?
The publisher keeps it at https://github.com/alirezarezvani/claude-skills.
Which agents can use ab-test-setup?
Claude, Cursor, and Codex can read a SKILL.md you add from the publisher's repository.
When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
name: "ab-test-setup"
description: When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: marketing
updated: 2026-03-06
A/B Test Setup
You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.
Initial Assessment
Check for product marketing context first:
If .claude/product-marketing-context.md exists, read it before asking questions. Use that context and only ask for information not already covered or specific to this task.
Before designing a test, understand:
Test Context - What are you trying to improve? What change are you considering?
Current State - Baseline conversion rate? Current traffic volume?
Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].
Example
Weak: "Changing the button color might increase clicks."
Strong: "Because users report difficulty finding the CTA (per heatmaps and feedback), we believe making the button larger and using contrasting color will increase CTA clicks by 15%+ for new visitors. We'll measure click-through rate from page view to signup start."
Test Types
Type
Description
Traffic Needed
A/B
Two versions, single change
Moderate
A/B/n
Multiple variants
Higher
MVT
Multiple changes in combinations
Very high
Split URL
Different URLs for variants
Moderate
Sample Size
Calculate It (bundled tool)
Use this skill's own calculator — don't eyeball it:
Paste sample_size_per_variation and the duration estimate directly into the test plan's "Sample size + duration" row before any test is approved to run.
Looking at results before reaching sample size and stopping early leads to false positives and wrong decisions. Pre-commit to sample size and trust the process.
Ranked experiments by expected impact and feasibility
Communication
All outputs should meet the quality standard: clear hypothesis, pre-registered metrics, and documented decisions. Avoid presenting inconclusive results as wins. Every test should produce a learning, even if the variant loses. Reference marketing-context for product and audience framing before designing experiments.
Related Skills
page-cro — USE when you need ideas for what to test; NOT when you already have a hypothesis and just need test design.
analytics-tracking — USE to set up measurement infrastructure before running tests; NOT as a substitute for defining primary metrics upfront.
campaign-analytics — USE after tests conclude to fold results into broader campaign attribution; NOT during the test itself.
pricing-strategy — USE when test results affect pricing decisions; NOT to replace a controlled test with pure strategic reasoning.
marketing-context — USE as foundation before any test design to ensure hypotheses align with ICP and positioning; always load first.
How do I install ab-test-setup?
Download .gemini/skills/ab-test-setup/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
Where does ab-test-setup live?
The publisher keeps it at https://github.com/alirezarezvani/claude-skills.
Which agents can use ab-test-setup?
Claude, Cursor, and Codex can read a SKILL.md you add from the publisher's repository.