CRO Glossary
Experiment Roadblock
An organizational, technical, or political obstacle that delays or prevents an experiment from launching or shipping its winning variant.
An experiment roadblock is anything that stops a well-designed test from actually running or from having its results acted on. This is distinct from statistical problems like low power or contamination — roadblocks are usually about people, process, or engineering capacity rather than the data itself. Common examples include an engineering team deprioritizing the QA needed to ship a variant, a stakeholder vetoing a design change late in the process, or a legal/compliance review that stalls a pricing test for weeks.
Practitioners care about roadblocks because they quietly kill experimentation programs even when the testing methodology is sound. A team can run flawless A/B tests and still fail to grow if every winning result gets stuck waiting for a sprint slot, or if a director overrides a statistically significant result because it conflicts with a personal preference. Tracking roadblocks over time (who blocked what, and why) helps a CRO team build the internal case for more autonomy or faster ship cycles.
A concrete example: a growth team finds a 12% lift from a simplified checkout flow, but the change requires backend work owned by a different team with no capacity for two quarters. The experiment is 'won' analytically but blocked operationally — the roadblock, not the data, becomes the bottleneck to realizing the uplift.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply experiment roadblock.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.