CRO Glossary
Instrumentation
The tracking code and event logging infrastructure that captures user actions so experiments can be measured accurately.
Instrumentation is the plumbing beneath every experiment: the analytics events, tags, and logging that record what a user actually did — clicked a button, viewed a page, completed a purchase — so that conversion rates and downstream metrics can be calculated at all. If instrumentation is broken or incomplete, no amount of statistical rigor in the analysis will save the test, because the underlying data is wrong.
Common instrumentation problems include double-firing events (inflating conversion counts), missing events on certain browsers or devices (undercounting a segment), and events firing before a page has fully loaded (creating SRM between control and variant if one version loads faster). These issues are often invisible until someone reconciles experiment numbers against a trusted source like backend order data.
Before trusting any experiment result, experienced teams QA the instrumentation by manually walking through both control and variant, checking that every step of the conversion funnel fires exactly once per real action, on every major browser and device the test targets. Good instrumentation hygiene is unglamorous but is consistently the difference between a trustworthy testing program and one that produces results nobody believes.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply instrumentation.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.