CRO Glossary
Experiment Velocity
The rate at which a team designs, launches, and concludes experiments over a given period.
Experiment velocity is a program-level metric — often expressed as tests shipped per month or per quarter — used to measure how efficiently an organization is learning through experimentation. High velocity generally correlates with faster compounding gains, but only if quality is held constant; a team that rushes tests without adequate test duration or statistical power can post an impressive velocity number while generating unreliable, non-repeatable results.
Velocity is most useful as a leading indicator of program health and organizational buy-in (are there enough engineers, enough traffic, enough decision-makers signed off to keep tests moving), rather than as a proxy for impact. Many mature CRO teams track velocity alongside win rate and average lift per test, since a program can have high velocity and low win rate, or vice versa, and each combination suggests a different intervention — more ideation, better prioritization, or more rigorous QA.
Example: a team increases experiment velocity from 2 tests per month to 8 by building a self-serve experimentation platform, but discovers their win rate drops because many of the new tests are underpowered. They then set a minimum sample size policy to keep velocity high without sacrificing reliability.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
See this in the wild
ABWatcher watches how top teams apply experiment velocity.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.