CRO Glossary
Effect Size
The magnitude of the difference between a variant and control, expressed in absolute or relative terms.
Effect size answers 'how big was the change,' as opposed to statistical significance, which answers 'how confident are we that the change is real.' A test can be statistically significant with a tiny effect size, or show a huge effect size that isn't statistically significant because the sample was too small. Practitioners need both numbers to make good decisions.
Effect size is usually reported as an absolute difference (control converts at 4.0%, variant at 4.6%, a 0.6 percentage point lift) or a relative difference (a 15% relative lift). Relative lift is often more useful for comparing tests across different baseline conversion rates, while absolute lift is more useful for estimating real business impact like additional revenue.
Effect size is also the key input to a minimum detectable effect calculation before a test starts: you decide the smallest effect size that would be worth detecting, and that determines the sample size you need. A test that later observes an effect size below that threshold, even if directionally positive, generally shouldn't be treated as a meaningful win.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
See this in the wild
ABWatcher watches how top teams apply effect size.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.