- name
SKILL.md
技能檔案保留在發布者的儲存庫中。
在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add sickn33/agentic-awesome-skills --skill ab-test-setup -a cursor -y在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add sickn33/agentic-awesome-skills --skill ab-test-setup -a claude-code -y在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add sickn33/agentic-awesome-skills --skill ab-test-setup -a codex -y在終端機中貼上複製的指令。它會將這個 SKILL.md 安裝到對應的代理中。
npx skills add sickn33/agentic-awesome-skills --skill ab-test-setup -a grok -y在 Grok Bot 聊天中傳送複製的訊息。Bot 會從 GitHub 取得這個 SKILL.md,並加入你所有 Bot 共用的技能中。
Install this skill: https://github.com/sickn33/agentic-awesome-skills/tree/HEAD/plugins/agentic-awesome-skills-claude/skills/ab-test-setup來自發布者
Define an experiment that can answer a specific product question, and verify its assumptions before exposing users. This procedure cannot guarantee validity by itself.
You must have:
A valid hypothesis includes:
Before designing variants or metrics, you MUST:
Use the hypothesis already agreed in the task. If a launch-critical choice is missing, present the concrete choice for confirmation while continuing independent analysis. Do not repeatedly request approval for a decision already authorized.
Explicitly list assumptions about:
If assumptions are weak or violated:
Choose the simplest valid test:
Default to A/B unless there is a clear reason otherwise.
Define upfront:
Estimate:
Do NOT proceed without a realistic sample size estimate.
Before entering the Execution Readiness Gate below, run through this checklist to make "Tracking is verified" mean something concrete:
If any of the above fails, stop and resolve it before Gate 8.
You may proceed to implementation only if all are true:
If any item is missing, stop and resolve it.
DO:
DO NOT:
When interpreting results:
Document:
Store records in a shared, searchable location to avoid repeated failures.
Refuse to proceed if:
Explain why and recommend next steps.
Use when a product change has enough eligible traffic for a randomized comparison and a measurable outcome. For low-volume launches or qualitative discovery, consider usability research or descriptive measurement instead of claiming causal lift.
For an illustrative binary metric, estimate the per-variant sample for a change from 10% to 11% (one percentage point, 10% relative lift), 50/50 allocation, two-sided alpha 0.05 and power 0.80. This Python 3 large-sample approximation uses Cohen's proportion effect size:
from math import asin, ceil, sqrt
from statistics import NormalDist
baseline, variant = 0.10, 0.11 # illustrative assumptions, not measured data
alpha, power = 0.05, 0.80
h = abs(2 * asin(sqrt(variant)) - 2 * asin(sqrt(baseline)))
z = NormalDist()
per_variant = ceil(2 * (z.inv_cdf(1 - alpha / 2) + z.inv_cdf(power)) ** 2 / h ** 2)
print(per_variant)
Expected output: 14745 observations per variant for these assumptions.
This calculation assumes independent units, one binary outcome, a fixed horizon and no multiplicity adjustment. It is inappropriate for clustered or repeated observations, sequential decisions or continuous revenue metrics. Account for eligible traffic, attrition, outcome delay and the sampling unit before turning a sample estimate into calendar duration. Equal assumed rates have zero effect size and no finite sample for detecting that difference.
Observation: users abandon a long signup form.
Change: remove one optional field; unit: account; allocation: 50/50 and stable.
Primary metric: completed signup / eligible assigned accounts within 24 hours.
Guardrails: validation failures and support requests.
Before launch: estimate sample needs from baseline and MDE, verify exposure and
completion IDs, define analysis window and stopping rule.
Expected report: counts, absolute/relative effect, interval, data-quality checks,
guardrail results and a decision with its limits; never just “p < 0.05, ship”.
Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking, and predeclare metrics and stopping rules.
---name: ab-test-setupdescription: Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size…---# plugins/agentic-awesome-skills-claude/skills/ab-test-setup/SKILL.md常見問題
從發布者下載 plugins/agentic-awesome-skills-claude/skills/ab-test-setup/SKILL.md,或複製路徑後開啟儲存庫。PlugMCP 只收錄這項技能,不代管檔案。
發布者將它存放在 https://github.com/sickn33/agentic-awesome-skills。
Claude、Cursor、Codex 和 Grok Build 可以讀取你從發布者儲存庫加入的 SKILL.md。在聊天中傳送技能的 GitHub 資料夾連結,Grok Bot 就會安裝它。