CRO Glossary
Variant Fatigue
The gradual decline in a test variant's performance as the novelty or attention it initially captured wears off.
Variant fatigue describes a pattern where a new design, message, or offer performs strongly in the first days of a test and then settles into a lower, more "true" performance level as returning visitors get used to it. It's closely related to the novelty effect, but variant fatigue specifically refers to the decay curve observed within a single test's lifespan rather than the broader phenomenon of novelty inflating results.
This matters because teams that call a winner too early — based on a strong first few days — can roll out a change that looked great initially but converges toward parity (or even underperforms) once fatigue sets in. It's especially relevant for tests involving banners, popups, promotional messaging, or anything that relies on grabbing attention through change itself rather than genuinely improving usability or value.
A concrete example: a retailer tests a bright, animated "Sale Ends Soon" banner against a static control. The animated version drives a 15% lift in click-through in the first three days, but by day ten the lift has shrunk to 2% as repeat visitors stop noticing it. Cutting the test at day three would have overstated the true long-term effect.
To guard against variant fatigue, practitioners run tests for a full pre-defined test duration that spans multiple visitor cohorts and business cycles, and they check whether the lift trend is flat, growing, or declining over time rather than relying on a single end-of-test snapshot.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply variant fatigue.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.