CRO Glossary
Test Idea Backlog Scoring
The practice of ranking proposed experiments using a structured framework so teams prioritise the highest-value tests first.
Every experimentation program eventually accumulates more test ideas than it can run. Test idea scoring is the discipline of evaluating each proposed experiment against consistent criteria—like expected impact, confidence, and effort—so the team runs the most promising tests first instead of whichever idea is loudest in the room.
This matters because experimentation capacity is finite: traffic, engineering time, and QA bandwidth all cap how many tests can run in parallel. Without scoring, teams tend to gravitate toward easy-to-build but low-impact tests (a button color tweak) over harder, higher-leverage ones (a checkout flow redesign). Scoring frameworks force an explicit tradeoff conversation and create a defensible, repeatable prioritization process that survives team turnover.
For example, a growth team might score every backlog item on a 1-5 scale for potential impact, confidence in the hypothesis, and ease of implementation, then multiply or average those scores to get a ranked list. The team commits to running the top-scored tests next quarter, revisiting scores as new data or business priorities emerge.
Scoring isn't purely mechanical—teams often override scores for strategic reasons, like testing a foundational hypothesis before building on it. The value is in making prioritization visible and debatable rather than ad hoc.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply test idea backlog scoring.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.