CRO Glossary
Wait Time (Time to Significance)
The elapsed time an experiment must run before it collects enough data to reach a statistically valid conclusion.
Wait time is the practical, calendar-time counterpart to sample size: it tells you how many days or weeks you'll actually need to leave a test live given your real traffic volume and baseline conversion rate. Two tests can have identical minimum detectable effects and required sample sizes, but wildly different wait times depending on whether the page gets 500 or 50,000 visits a day.
Teams care about this because it directly shapes what's feasible to test. A low-traffic checkout page trying to detect a 2% lift might require months of runtime, which is often longer than the business is willing to wait or than the product roadmap allows. In those cases, teams either raise the MDE (accept only detecting bigger effects), combine traffic across pages, or switch to a qualitative or analytics-based approach instead of a randomized test.
A common mistake is calculating sample size correctly but ignoring wait time until the test is already live, then panicking or stopping early when results aren't significant after a week. Estimating wait time up front, before launch, keeps expectations aligned and prevents premature peeking or resourcing conflicts with other planned experiments.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply wait time (time to significance).
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.