CRO Glossary
Randomization Unit
The entity (user, session, or device) that gets randomly assigned to a test variant.
The randomization unit is what actually gets split into control and treatment — typically a user ID, a cookie, a session, or sometimes an account or device. Choosing the wrong unit is one of the most common ways experiments get silently corrupted.
If you randomize by session but a user can have multiple sessions, the same person might see both the control and the treatment across different visits, which dilutes the measured effect and can bias results toward showing no difference. Conversely, randomizing by logged-in user ID keeps the experience consistent per person but can miss anonymous traffic that never logs in.
The randomization unit should match the level at which the intervention makes sense and the level at which you measure the outcome. A pricing page test usually randomizes by user (so someone doesn't see two different prices), while a backend infrastructure test might randomize by server or request. Getting this wrong is a common hidden cause of a SRM.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply randomization unit.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.