CRO Glossary
Bayesian Inference
A statistical approach that updates the probability a variant is better as data accumulates, rather than testing against a fixed null hypothesis.
Bayesian inference frames experiment results in terms of probability directly: "there's a 92% chance variant B has a higher conversion rate than A," rather than the frequentist framing of p-values and null hypotheses used in classic statistical significance testing. It starts with a prior belief about likely outcomes and updates that belief as data comes in, producing a probability distribution over the possible effect sizes.
Practitioners often prefer Bayesian methods for CRO because the output is more intuitive to non-statisticians (a direct probability of being better, plus an estimated range of the lift) and because well-designed Bayesian methods tolerate continuous monitoring better than naive frequentist significance testing, reducing the practical impact of the peeking problem.
For example, a Bayesian dashboard might report "Variant B has a 87% probability of beating Variant A, with an expected uplift of 4.2%, credible interval 1%–7%." This is often easier for a product manager to act on than a p-value, though the underlying uncertainty and the need for adequate sample size don't disappear just because the framing is more intuitive.
Related terms
Above vs Below Median Split
A segmentation technique that divides users into two groups at the median of a metric to compare high and low engagers.
Qualitative Feedback Loop
A recurring process of collecting and acting on user comments, session recordings, and survey responses to inform test hypotheses.
Experiment Metadata
The structured record of who, what, when, and how for a test — hypothesis, owner, dates, variants, and metrics used to track and audit it.
Pre-Test Analysis
The upfront work of validating traffic, metrics, and baseline data quality before launching an experiment.
Above-the-Fold vs Below-the-Fold Messaging Hierarchy
How the order and prominence of value-proposition messaging is split between the visible screen and the rest of the page.
Experiment Contamination
When a control or variant group is unintentionally exposed to the other condition, corrupting the comparison.
See this in the wild
ABWatcher watches how top teams apply bayesian inference.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.