Interaction Effect

An interaction effect happens when the effect of one variable on conversion depends on the level of another variable, rather than the two acting independently. This matters most in multivariate testing or when running several experiments concurrently on the same page: two changes that each look neutral or positive in isolation can combine to produce a strongly positive or negative result, and vice versa.

Practitioners care about interaction effects because ignoring them leads to false conclusions about which change 'caused' a lift. If you test a new headline and a new CTA color separately and both win, you might assume shipping both together doubles the benefit — but the two elements could visually clash or reinforce the same psychological trigger, producing a smaller (or negative) combined effect.

A concrete example: a bold red 'Buy Now' button wins on its own, and a scarcity message ('Only 3 left!') also wins on its own. But together, the aggressive red button plus scarcity copy feels pushy and increases bounce rate, so the combined variant underperforms both individual winners. Detecting this requires a full factorial multivariate test rather than two sequential A/B tests, since sequential tests can't reveal how variables behave when combined.

Because factorial designs need much larger sample sizes to detect interactions reliably, teams often only check for interaction effects on high-traffic pages or when two changes touch the same UI region or user decision point.