Metrics, Hypothesis & Minimum Detectable Effect
All lessons
- Metrics, Hypothesis & Minimum Detectable EffectFree
- Sample Size for a Conversion Test (80% Power) Pro
- Test Duration: From Sample Size to Full Weeks Pro
- Analysing a Conversion Test: z-Test, p-Value & Confidence Interval Pro
- Revenue per Visitor: Welch t-Test on Skewed Data Pro
- Many Variants: The Multiple-Comparison Problem & Bonferroni Pro
- Sample Ratio Mismatch: Chi-Square Check of the Traffic Split Pro
- Peeking & Sequential Testing: Why Early Stops Lie Pro
You are not signed in — the lab works, but your progress is not saved. Start free
Read first
Write it down before the start
- Hypothesis, primary metric, guardrails, MDE, significance level, power and duration go into the plan before the test starts.
- Pick one primary metric. With five "primary" metrics, one of them will look like a win by chance.
- Relative vs absolute: 3.00% → 3.24% is +8% relative but +0.24 pp absolute. Always say which one you mean.
Loading the spreadsheet…