All terms

CRO Glossary

Pre/Post Analysis

A method of estimating a change's impact by comparing metrics before and after launch, without a control group.

Pre/post analysis compares a metric in the period before a change shipped to the period after, and attributes the difference to the change. It's the fastest way to get a read on impact when you can't or didn't run a controlled experiment, but it's also the weakest form of evidence because it conflates the treatment effect with everything else that changed over time: seasonality, marketing spend, macro trends, competitor moves, or even the day of the week.

Teams reach for pre/post analysis when a feature was rolled out to 100% of users immediately, when an A/B test wasn't technically feasible, or when stakeholders want a quick sanity check before a more rigorous read-out arrives. It's genuinely useful as a triangulation signal alongside a proper experiment, but dangerous as a sole source of truth because any coincidental trend gets credited to the launch.

For example, a team redesigns checkout and sees conversion rate jump 8% the week after launch. Without a holdout group, they can't tell how much of that lift came from the redesign versus a simultaneous drop in payment gateway errors or a seasonal spike in demand. Analysts often try to patch this with difference-in-differences techniques, comparing the pre/post delta against a similar unaffected segment or market, which at least partially controls for shared external trends.

The core risk practitioners should flag: pre/post comparisons are especially vulnerable to regression to the mean and to any external event that happens to coincide with launch, so conclusions should be stated with much lower confidence than a randomized test.

Related terms

See this in the wild

ABWatcher watches how top teams apply pre/post analysis.

Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.