All terms

CRO Glossary

Evan Miller Sample Size

A common shorthand for using Evan Miller's online calculator to estimate the sample size needed for a valid A/B test.

Practitioners often refer casually to "running it through Evan Miller" as a stand-in for doing a proper pre-test power calculation. The tool takes a baseline conversion rate, a minimum detectable effect, statistical power, and significance threshold, and returns the number of visitors (or conversions) needed per variant before the test can reliably detect the effect you're looking for.

The reason this matters is that teams frequently launch tests without checking whether their traffic can actually support the effect size they hope to see. A test that needs 400,000 visitors per arm to detect a 2% relative lift will simply never reach significance on a page that gets 5,000 visitors a month — no matter how long it runs.

For example, a team redesigning a checkout page with a 3% baseline conversion rate might plug in an MDE of 10% relative lift, 80% power, and a 95% confidence level, and learn they need roughly 40,000 users per variant. If their site only gets 10,000 checkout visits a month, they now know upfront to either wait longer, target a bigger effect, or test on a higher-traffic page instead.

This kind of calculation is essentially the same math behind formal sample size calculators, but the name has become a genericized reference point in the industry because Miller's calculator was one of the first widely used, free, and simple tools for this purpose.

Related terms

See this in the wild

ABWatcher watches how top teams apply evan miller sample size.

Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.