CRO Glossary
Experiment Tracking Plan
A documented specification of what events, properties, and metrics an experiment needs instrumented before it launches.
An experiment tracking plan is the blueprint that connects a hypothesis to the actual data that will be collected to test it. It lists the events that need to fire (e.g., "checkout_started", "cta_clicked"), the properties attached to each event (variant ID, user ID, timestamp, device type), and which metrics — primary, secondary, and guardrail — will be computed from them. It's typically written before development begins, so engineers know exactly what to instrument and analysts know exactly what to expect in the data.
Teams care about this because a huge share of "failed" experiments aren't actually failures of the idea — they're failures of measurement. Missing events, inconsistent property naming, or events that fire twice silently corrupt results, and by the time anyone notices, the test window has often already closed. A tracking plan turns instrumentation into a checklist that can be reviewed and QA'd before traffic starts flowing, rather than discovered as a gap during analysis.
For example, before testing a new checkout flow, a tracking plan might specify that each step of the funnel fires a distinct event with the variant assignment attached, that failed payment attempts are captured separately from abandonments, and that the currency and cart value are logged on every transaction event so revenue-based metrics can be computed accurately across regions.
Well-run experimentation programs treat the tracking plan as a required artifact alongside the experiment design document — one defines what you're testing and why, the other defines exactly how you'll know what happened.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply experiment tracking plan.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.