CRO Glossary
Qualified Traffic
Visitors who match the intended audience and behavior profile for a test, as opposed to bots, spam, or irrelevant segments.
Qualified traffic is the subset of visitors whose inclusion in a test actually gives you a meaningful read on your hypothesis. Including unqualified traffic — bot crawlers, internal QA sessions, users from an unrelated geography, or people who bounce before ever seeing the tested element — adds noise and can silently bias results toward the null, making a real effect harder to detect. Practitioners filter for qualified traffic by setting clear inclusion criteria before launch: exclude known bot user agents and internal IP ranges, require a minimum session engagement, or scope the test to the specific landing page traffic source being optimized. This filtering should happen at the exposure and reporting layer consistently for both control and variant, otherwise you risk introducing exactly the kind of imbalance that shows up as SRM.
For example, if you're testing a checkout flow change aimed at mobile shoppers arriving from a paid social campcampaign, including desktop organic traffic in the analysis dilutes the effect and can mask a real win. Defining qualified traffic up front, in the hypothesis statement itself, avoids arguments after the fact about which users 'should' count.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply qualified traffic.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.