Qualified Traffic

Qualified traffic is the subset of visitors whose inclusion in a test actually gives you a meaningful read on your hypothesis. Including unqualified traffic — bot crawlers, internal QA sessions, users from an unrelated geography, or people who bounce before ever seeing the tested element — adds noise and can silently bias results toward the null, making a real effect harder to detect. Practitioners filter for qualified traffic by setting clear inclusion criteria before launch: exclude known bot user agents and internal IP ranges, require a minimum session engagement, or scope the test to the specific landing page traffic source being optimized. This filtering should happen at the exposure and reporting layer consistently for both control and variant, otherwise you risk introducing exactly the kind of imbalance that shows up as SRM.

For example, if you're testing a checkout flow change aimed at mobile shoppers arriving from a paid social campcampaign, including desktop organic traffic in the analysis dilutes the effect and can mask a real win. Defining qualified traffic up front, in the hypothesis statement itself, avoids arguments after the fact about which users 'should' count.