CRO Glossary
Minimum Detectable Effect (MDE)
The smallest true difference between variants that a test is designed to reliably detect.
The minimum detectable effect is the smallest lift (or drop) in your metric that an experiment is powered to catch given its sample size, duration, and statistical settings. Set it before launch, not after: it directly determines how long you need to run a test and how many visitors you need per variant. A smaller MDE means you can detect subtler effects, but it requires dramatically more traffic to do so reliably.
Teams often skip this step and just "run the test for two weeks," which quietly means they've accepted whatever MDE that sample size implies — sometimes an unrealistically large one, like a 30% conversion lift that will never happen from a button color change. Choosing MDE forces an honest conversation: is it worth running a test for six weeks to detect a 2% lift, or should you accept only detecting effects of 10% or more?
For example, a checkout page with 2% baseline conversion and 20,000 weekly visitors per variant might only be able to detect a 15% relative lift within two weeks at 80% statistical power. If the true effect of your change is a 5% lift, the test will likely end inconclusive — not because the change didn't work, but because it was underpowered to see it.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply minimum detectable effect (mde).
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.