Minimum Detectable Effect (MDE)

The minimum detectable effect is the smallest lift (or drop) in your metric that an experiment is powered to catch given its sample size, duration, and statistical settings. Set it before launch, not after: it directly determines how long you need to run a test and how many visitors you need per variant. A smaller MDE means you can detect subtler effects, but it requires dramatically more traffic to do so reliably.

Teams often skip this step and just "run the test for two weeks," which quietly means they've accepted whatever MDE that sample size implies — sometimes an unrealistically large one, like a 30% conversion lift that will never happen from a button color change. Choosing MDE forces an honest conversation: is it worth running a test for six weeks to detect a 2% lift, or should you accept only detecting effects of 10% or more?

For example, a checkout page with 2% baseline conversion and 20,000 weekly visitors per variant might only be able to detect a 15% relative lift within two weeks at 80% statistical power. If the true effect of your change is a 5% lift, the test will likely end inconclusive — not because the change didn't work, but because it was underpowered to see it.