CRO Glossary
A/A Test
A test that splits traffic between two identical experiences to validate that the testing system itself is working correctly.
An A/A test runs the same version of a page or feature against itself, splitting visitors into two (or more) groups exactly as you would in a real experiment, but without changing anything they see. Because both groups are identical, you expect no meaningful difference in conversion rate or other metrics between them.
Teams run A/A tests to validate their experimentation infrastructure before trusting it with real decisions. If an A/A test reports a statistically significant difference between two identical groups more often than your chosen significance threshold would predict, that's a red flag: it can point to bugs in randomization, tracking, or analysis code, or to a sample ratio mismatch.
For example, a company migrating to a new experimentation platform might run an A/A test for two weeks on their checkout flow. If the tool reports a 'winner' with 95% confidence even though both variants are the same code, that suggests the tool's stats engine, event logging, or bucketing logic needs debugging before it's used for real A/B tests.
A/A tests are also useful for estimating baseline variance and noise in a metric, which can help calibrate expectations for future minimum detectable effect calculations.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
See this in the wild
ABWatcher watches how top teams apply a/a test.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.