CRO Glossary
Network Effect Bias
A bias where a test's outcome is distorted because treatment and control users interact with or influence each other.
Network effect bias (sometimes called interference or spillover) happens when the outcome for a user in one arm of an experiment is affected by another user's assignment. Standard A/B test math assumes each unit's response is independent of every other unit's treatment, but on social platforms, marketplaces, messaging tools, or anything with sharing, referrals, or shared inventory, that assumption breaks down. If a control user receives a message from a treatment user who got a new invite flow, the control group is no longer a clean 'untouched' baseline.
Practitioners care because interference silently biases lift estimates, usually toward zero, making a genuinely effective feature look flat or even negative. It's especially dangerous in growth and virality experiments, two-sided marketplaces (buyers/sellers sharing inventory), and collaboration tools where users message or share content across arms.
A classic example: testing a new 'invite a friend' flow. If invited friends land in the opposite experiment arm from the inviter, engagement measured per-user conflates the treatment effect with contamination from the control arm. Common mitigations include cluster randomization (randomizing by friend-group, geography, or market rather than by individual), ego-network or graph-cluster designs, and switchback tests for marketplace supply/demand effects.
The practical takeaway: before trusting a null or muted result, ask whether users in different arms could plausibly interact, and if so, whether the randomization unit matches the level at which effects actually propagate.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
See this in the wild
ABWatcher watches how top teams apply network effect bias.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.