CRO Glossary
Personalization
Tailoring content, offers, or experiences to specific users or segments based on data like behavior, location, or lifecycle stage.
Personalization goes a step beyond a single-variant A/B test by serving different experiences to different user segments based on characteristics like geography, device, referral source, past purchase behavior, or account plan tier. Rather than asking 'which single version wins for everyone,' personalization asks 'which version wins for this type of user.'
Experimentation programs often discover personalization opportunities through segment analysis: a test might show a flat or negative result overall, but a strongly positive result within a specific segment, like mobile users or first-time visitors, that gets masked when looking only at the aggregate. That's a signal the flat result might really be two divergent stories, similar in spirit to Simpson's paradox, where an aggregate trend hides opposite patterns in subgroups.
A common example is showing returning customers a 'welcome back' message with their previously viewed products, while first-time visitors see a broader value proposition and trust signals. Because personalization multiplies the number of experiences being served and evaluated, teams need larger sample sizes per segment and clear rules for how segments are defined ahead of time, to avoid p-hacking through after-the-fact segment slicing.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply personalization.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.