All terms

CRO Glossary

Factorial Design

An experiment structure that tests two or more independent variables simultaneously in all possible combinations.

A factorial design lets you test multiple factors at once instead of running separate experiments for each one. If you have two headlines and two button colors, a full factorial design creates four variants covering every combination, so you can measure the effect of each factor plus whether they interact with each other. This is distinct from simply running sequential single-variable tests, because it reveals interaction effects that isolated tests would miss entirely.

Teams care about factorial designs because they are efficient: rather than running four separate A/B tests to learn about two variables, one factorial test can answer both questions with shared traffic. The tradeoff is statistical power — splitting traffic across more cells means each cell gets fewer visitors, so you need either more total traffic or a larger expected effect size to detect anything meaningful.

A concrete example: an ecommerce site wants to know if a shorter checkout form and a trust badge each improve completion rate, and whether they work better together or redundantly. A 2x2 factorial design (form length x badge presence) produces four groups and lets analysts estimate both main effects and the interaction in one coordinated test.

Factorial designs are closely related to multivariate testing but the term specifically emphasizes the combinatorial structure and the interest in interaction effects, not just the ability to test many elements.

Related terms

See this in the wild

ABWatcher watches how top teams apply factorial design.

Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.