All terms

CRO Glossary

Experiment Readiness Checklist

A pre-launch checklist confirming that an experiment's tracking, sample size, QA, and stakeholder sign-off are in place before it goes live.

An experiment readiness checklist is a gate teams run through before flipping an experiment on, meant to catch the avoidable mistakes that invalidate results after the fact — missing tracking events, an underpowered sample size, a broken variant on mobile, or no agreed-upon success metric. It exists because most experimentation failures aren't statistical; they're operational, caught too late to fix without restarting the test.

A typical checklist confirms: the hypothesis and primary metric are documented, the sample size and expected test duration have been calculated, instrumentation has been QA'd in a staging or preview environment, the randomization unit and target audience are correctly configured, guardrail metrics are defined, and someone has visually checked each variant across key devices and browsers. Many teams also require a second reviewer to sign off, similar to a code review, before launch.

For example, a team testing a new checkout flow might discover during readiness review that the analytics event for "payment submitted" fires on page load rather than on successful submission — a bug that would have silently inflated conversion counts in both arms and made the test's results meaningless. Catching this before launch, rather than during analysis, saves the cost of re-running the entire experiment.

Teams that skip this step tend to accumulate experiment debt: tests that have to be voided, re-run, or caveated heavily because something fundamental was wrong with the setup, undermining trust in the experimentation program as a whole.

Related terms

See this in the wild

ABWatcher watches how top teams apply experiment readiness checklist.

Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.