A/B Testing & Experimentation
Plan, size, run and analyse A/B tests in Excel: metrics and MDE, sample size and duration, z-tests, confidence intervals, revenue t-tests, Bonferroni, SRM checks and the peeking trap, in 8 graded labs.
The first lesson is free — no account needed.
About this course
An A/B testing course that takes one online shop's experiments from the test plan to the ship decision, in Excel. You set the primary, secondary and guardrail metrics and the minimum detectable effect, calculate the sample size per variant for 95% confidence and 80% power, turn it into a test duration in full weeks, analyse a conversion test with a z-test, p-value and confidence interval, test revenue per visitor with a Welch t-test, control false winners across several variants with a Bonferroni correction, check the traffic split for sample ratio mismatch with a chi-square test and measure how peeking inflates false positives. The first lab is free.
Lessons
- Metrics, Hypothesis & Minimum Detectable EffectFree
- Sample Size for a Conversion Test (80% Power) Pro
- Test Duration: From Sample Size to Full Weeks Pro
- Analysing a Conversion Test: z-Test, p-Value & Confidence Interval Pro
- Revenue per Visitor: Welch t-Test on Skewed Data Pro
- Many Variants: The Multiple-Comparison Problem & Bonferroni Pro
- Sample Ratio Mismatch: Chi-Square Check of the Traffic Split Pro
- Peeking & Sequential Testing: Why Early Stops Lie Pro