CRO Glossary
Pre-registration
Documenting a test's hypothesis, metrics, audience, and analysis plan before launch to prevent biased or cherry-picked conclusions.
Pre-registration means writing down, before you look at any data, exactly what you're testing, why, which metric counts as the primary success measure, how long the test will run, and how you'll analyze the results. It's borrowed from academic research, where it emerged as a defense against researchers quietly changing their hypothesis after seeing the results to make a finding look more impressive than it is.
In CRO, pre-registration matters because it's very easy to rationalize a result after the fact. If a test shows no lift on the primary metric but a secondary metric moved, it's tempting to declare victory on the secondary metric instead. If you wrote down beforehand that the primary metric was checkout conversion rate, you can't quietly swap in 'time on page' after the test disappoints. This discipline protects the integrity of your experimentation program and builds trust with stakeholders who review results.
A practical version of pre-registration is a one-page test brief: hypothesis, primary metric, guardrail metrics, minimum detectable effect, expected runtime, and randomization unit, all filled in and shared with the team before the experiment goes live. Some teams store these briefs in a shared doc or experimentation platform so nobody can edit the plan retroactively.
Without pre-registration, teams are vulnerable to p-hacking and post-hoc storytelling, where almost any test can be spun as a win by selectively highlighting whichever metric happened to move. Pre-registration doesn't guarantee a clean result, but it makes it much harder to fool yourself or your leadership.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
See this in the wild
ABWatcher watches how top teams apply pre-registration.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.