CRO Glossary
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Pre-test analysis is the diagnostic work a team does before an experiment goes live: checking that tracking fires correctly, that the baseline conversion rate is stable over time, that traffic volume actually supports the planned sample size, and that no seasonal or external event will contaminate the read. It's the difference between designing a test in a spreadsheet and knowing the test will actually produce a trustworthy answer.
Practitioners care about this because most "bad test" post-mortems trace back to something that was knowable in advance — a metric that was too noisy to move, an audience segment that was too small, or an instrumentation bug that silently undercounted conversions. Pre-test analysis catches these before they waste weeks of traffic and stakeholder patience.
A typical pre-test checklist includes pulling 4-8 weeks of historical data to check for trend or seasonality, confirming the metric's variance to sanity-check the MDE, running a dry-run or QA pass on the tracking implementation, and confirming that the randomization unit and eligible audience match what the hypothesis assumes. For example, before testing a new checkout flow, a team might discover in pre-test analysis that 15% of sessions never fire the checkout-start event on mobile Safari — fixing that first saves the whole experiment from being unusable.
Skipping this step is a common root cause of SRM, underpowered tests, and misleading read-outs that only surface after the test has already run its course.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
Metric Dilution
When a test's effect is real but gets averaged away because it only affects a small slice of the measured population.
See this in the wild
ABWatcher watches how top teams apply pre-test analysis.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.