CRO Glossary
Delta Method
A statistical technique for estimating the variance of a ratio metric (like conversion rate or revenue per visitor) so confidence intervals aren't biased.
Many core experimentation metrics — conversion rate, revenue per user, click-through rate — are ratios of two random variables (e.g., conversions divided by visitors, or revenue divided by sessions). The variance of a ratio isn't simply the variance of the numerator or denominator; naively computing standard errors on ratio metrics tends to understate true variance and produce overly narrow, overconfident confidence intervals. The delta method is a first-order Taylor expansion approach that approximates the variance of a function of random variables (like a ratio) from the variances and covariance of its components.
This matters most when the unit of randomization doesn't match the unit of analysis — for instance, when you randomize by user but measure revenue per session, and a single user can generate multiple sessions. In that case, sessions from the same user are correlated, and ignoring that correlation (treating each session as independent) makes significance tests overly optimistic. The delta method, often combined with clustering or bootstrapping, corrects for this.
In practice, most modern experimentation platforms apply the delta method automatically under the hood when computing confidence intervals for ratio metrics like 'revenue per visitor' or 'orders per session.' If you're building your own stats pipeline and only use naive variance formulas on ratios, you risk declaring false wins — the delta method (or a bootstrap equivalent) is the standard fix.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply delta method.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.