CRO Glossary
Randomization Unit
The entity (user, session, or device) that gets randomly assigned to a test variant.
The randomization unit is what actually gets split into control and treatment — typically a user ID, a cookie, a session, or sometimes an account or device. Choosing the wrong unit is one of the most common ways experiments get silently corrupted.
If you randomize by session but a user can have multiple sessions, the same person might see both the control and the treatment across different visits, which dilutes the measured effect and can bias results toward showing no difference. Conversely, randomizing by logged-in user ID keeps the experience consistent per person but can miss anonymous traffic that never logs in.
The randomization unit should match the level at which the intervention makes sense and the level at which you measure the outcome. A pricing page test usually randomizes by user (so someone doesn't see two different prices), while a backend infrastructure test might randomize by server or request. Getting this wrong is a common hidden cause of a SRM.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
See this in the wild
ABWatcher watches how top teams apply randomization unit.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.