CRO Glossary
Network Effect Bias
A bias where a test's outcome is distorted because treatment and control users interact with or influence each other.
Network effect bias (sometimes called interference or spillover) happens when the outcome for a user in one arm of an experiment is affected by another user's assignment. Standard A/B test math assumes each unit's response is independent of every other unit's treatment, but on social platforms, marketplaces, messaging tools, or anything with sharing, referrals, or shared inventory, that assumption breaks down. If a control user receives a message from a treatment user who got a new invite flow, the control group is no longer a clean 'untouched' baseline.
Practitioners care because interference silently biases lift estimates, usually toward zero, making a genuinely effective feature look flat or even negative. It's especially dangerous in growth and virality experiments, two-sided marketplaces (buyers/sellers sharing inventory), and collaboration tools where users message or share content across arms.
A classic example: testing a new 'invite a friend' flow. If invited friends land in the opposite experiment arm from the inviter, engagement measured per-user conflates the treatment effect with contamination from the control arm. Common mitigations include cluster randomization (randomizing by friend-group, geography, or market rather than by individual), ego-network or graph-cluster designs, and switchback tests for marketplace supply/demand effects.
The practical takeaway: before trusting a null or muted result, ask whether users in different arms could plausibly interact, and if so, whether the randomization unit matches the level at which effects actually propagate.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply network effect bias.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.