CRO Glossary
Experiment Velocity Metric
A measure of how many valid experiments a team ships and concludes over a given period.
Experiment velocity metric refers to the specific number used to track how fast a team is running tests — usually expressed as tests launched or tests concluded per week, month, or quarter. It's distinct from experiment quality: a team can run many tests quickly and still learn very little if those tests are poorly designed or underpowered. Most mature CRO programs track velocity alongside win rate and average effect size so that speed doesn't get optimized at the expense of rigor.
Practitioners care about this because experimentation programs tend to stall due to bottlenecks in design, engineering, QA, or analysis rather than a lack of ideas. Tracking the metric over time surfaces where the pipeline is actually slow — for example, if ideation is fast but engineering implementation takes three weeks, that's the constraint to fix, not the backlog of hypotheses.
A concrete example: a team logs that in Q1 they launched 12 experiments and concluded 9 with a clear read (win, loss, or inconclusive), giving a velocity of roughly 3 concluded tests per month. If leadership pushes for 6 per month without adding engineering resources, the team either cuts corners on QA or starts running underpowered tests — a trade-off worth surfacing explicitly rather than absorbing silently.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply experiment velocity metric.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.