CRO Glossary
Feature Flag
A code-level toggle that lets teams turn a feature or variant on or off for specific users without deploying new code.
A feature flag (or feature toggle) is a conditional switch in the codebase that determines whether a given user sees a new feature, design, or experience. Experimentation platforms build on top of feature flags to control traffic allocation — assigning some percentage of users to see variant A and the rest variant B — and to roll changes out gradually or roll them back instantly if something breaks, without needing a new deployment.
Beyond experimentation, flags are also used for staged rollouts (releasing a feature to 5% of users, then 25%, then 100%) and for kill switches during incidents. For CRO specifically, the important nuance is that flags must be evaluated consistently per randomization unit — the same user should keep seeing the same variant across sessions, or the test's data becomes unreliable.
Example: a team wraps a new checkout flow in a feature flag, allocates 50% of traffic to the new flow, and monitors guardrail metrics. When a payment bug is discovered mid-test, they flip the flag off for all users within minutes rather than waiting for an emergency deploy.
Related terms
Shrinkage Estimator
A statistical adjustment that pulls noisy per-variant or per-segment estimates toward an overall average to reduce overfitting to randomness.
Flicker Test
A diagnostic check to confirm whether users briefly see the original page before a variant renders in client-side experiments.
Pre-Experiment Bucketing Audit
A review process confirming users are being randomly and evenly assigned to experiment variants before launch.
Above-Fold Exclusion Rule
A test-design rule that deliberately prevents changes to above-the-fold elements so an experiment isolates below-the-fold effects.
Flicker-Free Testing
A technical setup that prevents users from briefly seeing the original page before a test variant loads.
Orthogonal Test Design
A test structure where multiple variables are changed independently so their individual effects can be measured without interference.
See this in the wild
ABWatcher watches how top teams apply feature flag.
Live A/B tests at 1,000+ high-converting brands, with plain-English hypothesis and takeaway.