CRO Glossary
Pre-registration
Documenting a test's hypothesis, metrics, audience, and analysis plan before launch to prevent biased or cherry-picked conclusions.
Pre-registration means writing down, before you look at any data, exactly what you're testing, why, which metric counts as the primary success measure, how long the test will run, and how you'll analyze the results. It's borrowed from academic research, where it emerged as a defense against researchers quietly changing their hypothesis after seeing the results to make a finding look more impressive than it is.
In CRO, pre-registration matters because it's very easy to rationalize a result after the fact. If a test shows no lift on the primary metric but a secondary metric moved, it's tempting to declare victory on the secondary metric instead. If you wrote down beforehand that the primary metric was checkout conversion rate, you can't quietly swap in 'time on page' after the test disappoints. This discipline protects the integrity of your experimentation program and builds trust with stakeholders who review results.
A practical version of pre-registration is a one-page test brief: hypothesis, primary metric, guardrail metrics, minimum detectable effect, expected runtime, and randomization unit, all filled in and shared with the team before the experiment goes live. Some teams store these briefs in a shared doc or experimentation platform so nobody can edit the plan retroactively.
Without pre-registration, teams are vulnerable to p-hacking and post-hoc storytelling, where almost any test can be spun as a win by selectively highlighting whichever metric happened to move. Pre-registration doesn't guarantee a clean result, but it makes it much harder to fool yourself or your leadership.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply pre-registration.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.