CRO Glossary
Experiment Cannibalization
When a winning variant improves one metric by pulling conversions away from another product area, channel, or metric rather than creating net-new value.
Experiment cannibalization happens when a change that appears to boost a target metric is actually just shifting demand from somewhere else in the business rather than generating incremental value. The headline result looks like a clear win, but a broader view reveals it came at the expense of another product line, page, or channel, so the net impact to the company is flat or even negative.
This matters because teams optimizing a single funnel or page in isolation can easily declare victory on a local metric while missing the bigger picture. It's especially common when a business has multiple products, tiers, or acquisition channels that compete for the same user attention or budget — an uplift in one place can simply be a transfer, not new growth.
A classic example: a retailer tests a homepage banner promoting a discounted product line and sees a strong lift in conversions for that line. Deeper analysis shows overall site revenue barely moved — customers who would have bought a full-price item instead bought the discounted one, and total average order value dropped. The experiment "won" on its primary metric while cannibalizing revenue elsewhere.
To catch cannibalization, teams pair the primary success metric with guardrail and revenue-level metrics that span the whole business, not just the tested surface, and sometimes run holdout groups at a broader scope (e.g., whole-account or whole-channel) to see if gains are truly incremental before rolling a change out fully.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply experiment cannibalization.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.