CRO Glossary
Baseline Drift
A gradual change in a metric's normal level over time that can be mistaken for a treatment effect if not accounted for.
Baseline drift refers to the way a site's or app's underlying conversion rate, engagement level, or other key metric shifts over time due to factors unrelated to any specific experiment — seasonality, marketing campaigns, product changes, or shifting traffic mix. If an experiment runs during a period of drift, the comparison between control and variant can be distorted even though both groups are running concurrently and randomization is intact.
This matters because teams often compare a live experiment's control group performance to a historical average rather than to the variant running at the same time. If the baseline has drifted upward or downward since that historical number was calculated, the experiment's lift can look artificially large or small, or a flat result can be wrongly attributed to the treatment when it was really the overall traffic mix changing.
The standard defense is to always compare control and variant from the same concurrent time window rather than against historical benchmarks, and to watch for drift within the control group itself across the test's duration — a steadily rising or falling control conversion rate mid-experiment is a signal that something external is moving, not just random noise.
For example, if a pricing page test runs through a national holiday sale, both control and variant will show inflated conversion rates due to the sale, not the experiment. A team that only looked at the variant's absolute lift versus last month's baseline — instead of versus the concurrent control — could wrongly conclude the new pricing layout was a huge win.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply baseline drift.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.