All terms

CRO Glossary

Carryover Effect

Bias in a test's results caused by residual influence from a prior experiment or treatment that hasn't fully worn off.

A carryover effect happens when users who were exposed to a previous experiment still behave differently because of that exposure, even after the earlier test has ended and a new one has begun. Instead of measuring the new treatment in isolation, you end up measuring some blend of the old and new effects, which can inflate, deflate, or completely mask the true impact of what you're currently testing.

This is a common risk when teams reuse the same audience pool across back-to-back experiments without a cooldown period, or when cookies and local storage retain a user's prior variant assignment. It's especially problematic for tests involving pricing, onboarding flows, or anything that changes a user's mental model of the product, since those effects can linger for days or weeks after exposure.

For example, if you ran a test last month that trained users to expect a discount banner, and this month you test a new checkout flow on the same returning visitors, some of the checkout lift or drop you observe may actually be residual behavior from the discount test, not the checkout change itself.

Practitioners guard against carryover by resetting or excluding recently-exposed users, spacing out sequential experiments on overlapping populations, and checking new-visitor-only segments as a sanity check against returning-visitor segments.

Related terms

See this in the wild

ABWatcher watches how top teams apply carryover effect.

Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.