ADVERTISEMENT
Google AdSense Slot (headerLeaderboard)
Set your Publisher ID in src/config/ads.ts to display live ads.
Data Science & Statistical Analysis• 6 min read
A/B Testing & Sample Size Calculation
Design scientifically valid online experiments, avoid peeking problems, and determine required sample sizes.
Avoid the "Peeking Problem"
Stopping an A/B test as soon as p < 0.05 on day 2 is a statistical fallacy known as the "peeking problem" that drastically inflates false positive rates. Always predetermine sample size before launching.
ADVERTISEMENT
Google AdSense Slot (inContentBanner)
Set your Publisher ID in src/config/ads.ts to display live ads.
Test Yourself With Exercise
Why is stopping an A/B test the instant it hits p < 0.05 without meeting predetermined sample size problematic?