CRO Glossary
Pre-test QA
The process of verifying an experiment's tracking, targeting, and variants work correctly before launching it to real traffic.
Pre-test QA is the checklist step that happens after an experiment is built but before it goes live. It covers confirming that each variant renders correctly across browsers and devices, that the right audience is being targeted, that analytics events fire as expected, and that the randomization logic is actually splitting traffic the way it's supposed to. Skipping this step is one of the most common causes of ruined experiments.
Teams care about pre-test QA because bugs caught after launch are far more expensive than bugs caught before. A broken tracking pixel, a variant that fails to load on Safari, or a redirect that leaks control users into the treatment group can silently invalidate weeks of data collection. Many of the problems downstream analysts spend hours diagnosing — sample ratio mismatch, inflated or deflated conversion rates, contaminated segments — trace back to something that a five-minute QA pass would have caught.
A typical pre-test QA pass includes: viewing each variant on desktop and mobile, checking that the experiment doesn't visually break with ad blockers or slow connections, firing a test conversion and confirming it logs to the right variant, and verifying the experiment is excluded from being shown to internal team members or bots. Some teams use a staging environment or a QA parameter in the URL to preview variants without being bucketed into the live experiment.
For example, before launching a checkout page redesign test, a team might run through the flow in incognito mode for each variant, confirm the new 'Apply Coupon' button triggers the same backend call as the old one, and check that the experiment correctly excludes users who already completed checkout once before the test started.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply pre-test qa.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.