Tabellab

Difference-in-Differences: Pre and Test Averages, Change per Group, Lift and Incremental Orders

Lesson 5 of 20 Lab 🔒 Locked

Lessons in this course

  1. Test vs Holdout Markets: Orders per 100k Residents, Group Rates, Relative Lift and Incremental OrdersLabFree
  2. Platform ROAS vs Incremental ROAS: Reported Revenue, Test-Measured Revenue and the Over-Reporting MultipleLab Locked
  3. Matched Markets: Weekly Correlation, Size Ratio, a Match Score and the Best Control CandidateLab Locked
  4. Pre-Period Fit: Scale Factor, Predicted Test Orders, Absolute Percentage Error and MAPELab Locked
  5. Difference-in-Differences: Pre and Test Averages, Change per Group, Lift and Incremental OrdersLab Locked
  6. Regression Counterfactual: Slope, R², FORECAST.LINEAR Predictions and Weekly and Total Incremental OrdersLab Locked
  7. Reading the Lift: Weekly Incremental Orders, Relative Lift, the Cumulative Curve and Overall LiftLab Locked
  8. Incremental Revenue, iROAS, Cost per Incremental Order, Break-Even iROAS and Incremental ProfitLab Locked
  9. Confidence Interval for the Lift: Residual Standard Error, Standard Error of the Total and a 95 % IntervalLab Locked
  10. Is the Lift Real? t-Statistics, Two-Sided p-Values, Decisions at a Chosen α and Interval Lower BoundsLab Locked
  11. Placebo Tests: Fake Lifts in the Pre-Period, a 95th-Percentile Threshold, Verdict and a Flag RuleLab Locked
  12. Test Duration: Standard Error by Week, Minimum Detectable Effect, a Duration Verdict and the Weeks NeededLab Locked
  13. Sizing the Test Budget: Expected Incremental Orders, Expected Lift, Detectability, Net Test Cost and the Minimum SpendLab Locked
  14. How Big a Holdout? Test and Holdout Markets, Standard Error, MDE, the Cost of Going Dark and the Cheapest Adequate DesignLab Locked
  15. Spillover & Contamination: Exposure of Control Markets, a Contamination Flag, Weighted Exposure and a Leakage-Adjusted LiftLab Locked
  16. Where Does It Work? Incremental Orders and Revenue by Market, SUMIFS by Market Type, iROAS per Type and a ChartLab Locked
  17. The Response Curve: Multi-Cell Spend Test, Incremental Orders, iROAS and Marginal iROAS Against Break-EvenLab Locked
  18. Calibrating Attribution: Calibration Factors from Geo Tests, Calibrated Revenue and ROAS, and How Rankings ChangeLab Locked
  19. Combining Several Tests: Inverse-Variance Weights, Recency Discount, a Pooled iROAS and the Pooled Standard ErrorLab Locked
  20. Final Project: A Complete Geo-Test Readout with Counterfactual, Incremental Orders, iROAS, Interval, Decision and ChartLab Locked