CRO Glossary
Winsorization
A statistical technique that caps extreme values at a set percentile instead of removing them, to reduce outlier distortion while preserving sample size.
Winsorization addresses the same core problem as outlier trimming — extreme values skewing an experiment's results — but takes a gentler approach. Instead of deleting outlier observations entirely, it replaces values above (or below) a chosen threshold, such as the 99th percentile, with the value at that threshold. This keeps every user or session in the analysis while limiting how much any single extreme data point can influence the mean or variance.
This matters most for metrics with heavy right tails, like revenue per visitor or session duration, where a handful of whale purchases or bot sessions can dominate an average and make a test look like it won or lost based on a few data points rather than a genuine behavioral shift. Winsorizing stabilizes these metrics so that statistical tests reflect the typical user experience rather than being hostage to rare extremes, which is especially important in e-commerce and B2B experiments with high-value but infrequent transactions.
For example, if an experiment measures average order value and one customer places a $50,000 bulk order during the test window, winsorizing at the 99th percentile would cap that order's contribution to roughly what the 99th-percentile order value is, rather than letting it single-handedly swing the result. Analysts typically decide the winsorization threshold before looking at results, to avoid the appearance of cherry-picking a cutoff that produces a favorable outcome — a discipline closely related to pre-registration.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply winsorization.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.