Incrementality Testing: Geo Holdouts, Matched Markets, Pre-Period Fit, Lift & Confidence, Incremental ROAS, Test Sizing & Attribution Calibration
Measure what advertising really causes. You will compare test and holdout markets, set platform ROAS beside incremental ROAS, choose matched control markets, check the pre-period fit, use difference-in-differences and a regression counterfactual, read weekly and cumulative lift, value it as iROAS and incremental profit against break-even, build confidence intervals, p-values and placebo tests, size test duration, budget and holdout, correct for spillover, split results by market type, read a spend response curve, calibrate attribution with test results and pool several tests, and finish with a complete test readout for Halvern Living, with 20 hands-on labs, fictional data and every cell checked as you go.
- One-time payment
- Instant access right after payment
- Secure checkout with Stripe
About this course
This advanced course teaches incrementality testing with geo holdouts through the numbers that decide it. In 20 interactive spreadsheet labs you work with the test results of Halvern Living, a fictional home and garden shop: you compare test and holdout markets on orders per 100,000 residents, set platform ROAS beside incremental ROAS, choose matched control markets by correlation and size, check the pre-period fit and build counterfactuals with difference-in-differences and regression. You then read weekly and cumulative lift, value it against break-even margin, add confidence intervals, p-values and placebo tests, size the duration, budget and holdout of a test, adjust for spillover, split lift by market type, read a spend response curve, calibrate attribution with test results, pool several tests and finish with a complete test readout and decision. Every formula is checked as you type. All companies, markets and data are fictional and illustrative.
Lessons
- Test vs Holdout Markets: Orders per 100k Residents, Group Rates, Relative Lift and Incremental OrdersFree
- Platform ROAS vs Incremental ROAS: Reported Revenue, Test-Measured Revenue and the Over-Reporting Multiple Locked
- Matched Markets: Weekly Correlation, Size Ratio, a Match Score and the Best Control Candidate Locked
- Pre-Period Fit: Scale Factor, Predicted Test Orders, Absolute Percentage Error and MAPE Locked
- Difference-in-Differences: Pre and Test Averages, Change per Group, Lift and Incremental Orders Locked
- Regression Counterfactual: Slope, R², FORECAST.LINEAR Predictions and Weekly and Total Incremental Orders Locked
- Reading the Lift: Weekly Incremental Orders, Relative Lift, the Cumulative Curve and Overall Lift Locked
- Incremental Revenue, iROAS, Cost per Incremental Order, Break-Even iROAS and Incremental Profit Locked
- Confidence Interval for the Lift: Residual Standard Error, Standard Error of the Total and a 95 % Interval Locked
- Is the Lift Real? t-Statistics, Two-Sided p-Values, Decisions at a Chosen α and Interval Lower Bounds Locked
- Placebo Tests: Fake Lifts in the Pre-Period, a 95th-Percentile Threshold, Verdict and a Flag Rule Locked
- Test Duration: Standard Error by Week, Minimum Detectable Effect, a Duration Verdict and the Weeks Needed Locked
- Sizing the Test Budget: Expected Incremental Orders, Expected Lift, Detectability, Net Test Cost and the Minimum Spend Locked
- How Big a Holdout? Test and Holdout Markets, Standard Error, MDE, the Cost of Going Dark and the Cheapest Adequate Design Locked
- Spillover & Contamination: Exposure of Control Markets, a Contamination Flag, Weighted Exposure and a Leakage-Adjusted Lift Locked
- Where Does It Work? Incremental Orders and Revenue by Market, SUMIFS by Market Type, iROAS per Type and a Chart Locked
- The Response Curve: Multi-Cell Spend Test, Incremental Orders, iROAS and Marginal iROAS Against Break-Even Locked
- Calibrating Attribution: Calibration Factors from Geo Tests, Calibrated Revenue and ROAS, and How Rankings Change Locked
- Combining Several Tests: Inverse-Variance Weights, Recency Discount, a Pooled iROAS and the Pooled Standard Error Locked
- Final Project: A Complete Geo-Test Readout with Counterfactual, Incremental Orders, iROAS, Interval, Decision and Chart Locked
Learner reviews
Be the first to review this course.