All terms

CRO Glossary

Test Cannibalization Risk

The risk that a winning variant in one experiment shifts behavior in a way that erodes or distorts the results of a related experiment.

Test cannibalization risk refers to the possibility that an experiment's effect isn't a net gain for the business but rather a shift of behavior from one metric, channel, or page to another that is being measured separately. It's distinct from simple test contamination (users seeing overlapping variants); this is about the underlying user behavior itself moving around in a way that muddies interpretation.

This matters because teams often optimize individual pages or steps in isolation. A test that increases clicks on a homepage promotion might simply be pulling conversions forward from a different acquisition channel or a later-funnel test, rather than creating incremental revenue. If that offsetting effect isn't tracked, the team celebrates a lift that never shows up in top-line numbers.

A concrete example: an experiment that adds an aggressive discount banner to the homepage boosts homepage-to-cart conversion, but a concurrent email campaign test in the same period sees its conversion rate drop because the same users are converting via the banner instead of the email link. Each test looks fine in isolation, but combined they've just reallocated the same pool of buyers rather than growing overall conversion.

Guarding against this usually means looking at a shared north star or revenue metric across concurrent experiments, not just each test's local success metric, and being cautious about running multiple tests that target overlapping user intents at the same time.

Related terms

See this in the wild

ABWatcher watches how top teams apply test cannibalization risk.

Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.