CRO Glossary
Interaction Effect
When the combined impact of two or more variables tested together differs from what you'd predict by summing their individual effects.
An interaction effect happens when the effect of one variable on conversion depends on the level of another variable, rather than the two acting independently. This matters most in multivariate testing or when running several experiments concurrently on the same page: two changes that each look neutral or positive in isolation can combine to produce a strongly positive or negative result, and vice versa.
Practitioners care about interaction effects because ignoring them leads to false conclusions about which change 'caused' a lift. If you test a new headline and a new CTA color separately and both win, you might assume shipping both together doubles the benefit — but the two elements could visually clash or reinforce the same psychological trigger, producing a smaller (or negative) combined effect.
A concrete example: a bold red 'Buy Now' button wins on its own, and a scarcity message ('Only 3 left!') also wins on its own. But together, the aggressive red button plus scarcity copy feels pushy and increases bounce rate, so the combined variant underperforms both individual winners. Detecting this requires a full factorial multivariate test rather than two sequential A/B tests, since sequential tests can't reveal how variables behave when combined.
Because factorial designs need much larger sample sizes to detect interactions reliably, teams often only check for interaction effects on high-traffic pages or when two changes touch the same UI region or user decision point.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply interaction effect.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.