CRO Glossary
Burn-In Period
An initial stretch of a running experiment whose early, often volatile data is excluded from analysis to avoid skewed conclusions.
A burn-in period is the window right after an experiment launches where results are treated as provisional and typically excluded from the final read-out. During this time, caching layers are still warming up, bots and QA traffic may be disproportionately represented, and early adopters who click on anything new can distort the numbers before the system settles into a steady state.
Practitioners care about this because the first hours or days of a test often look nothing like the rest of it. A variant might show a huge spike or dip simply because the sample is small and skewed toward unusual visitors, not because the treatment is actually working or failing. Cutting a test short based on burn-in data is a common way teams fool themselves into premature calls.
For example, if a team launches a new checkout flow on a Tuesday afternoon, the first few hours might be dominated by internal staff checking the rollout, inflating conversion rates in a misleading way. A sensible practice is to exclude the first 24-48 hours, or until traffic composition and conversion rates stabilize, before counting data toward the final decision.
Burn-in is related to but distinct from novelty effects: burn-in is about data hygiene and system stabilization, while novelty effects concern genuine but temporary behavioral responses to something new.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply burn-in period.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.