CRO Glossary
Factorial Design
An experiment structure that tests two or more independent variables simultaneously in all possible combinations.
A factorial design lets you test multiple factors at once instead of running separate experiments for each one. If you have two headlines and two button colors, a full factorial design creates four variants covering every combination, so you can measure the effect of each factor plus whether they interact with each other. This is distinct from simply running sequential single-variable tests, because it reveals interaction effects that isolated tests would miss entirely.
Teams care about factorial designs because they are efficient: rather than running four separate A/B tests to learn about two variables, one factorial test can answer both questions with shared traffic. The tradeoff is statistical power — splitting traffic across more cells means each cell gets fewer visitors, so you need either more total traffic or a larger expected effect size to detect anything meaningful.
A concrete example: an ecommerce site wants to know if a shorter checkout form and a trust badge each improve completion rate, and whether they work better together or redundantly. A 2x2 factorial design (form length x badge presence) produces four groups and lets analysts estimate both main effects and the interaction in one coordinated test.
Factorial designs are closely related to multivariate testing but the term specifically emphasizes the combinatorial structure and the interest in interaction effects, not just the ability to test many elements.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply factorial design.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.