CUPED Variance Reduction in A/B Testing, Explained (2026)

Most advice on speeding up A/B tests is about getting more traffic. CUPED is the opposite idea - get more signal out of the traffic you already have. It is one of the highest-leverage techniques in modern experimentation, and it is barely known outside the teams that live in their stats engine. CUPED can roughly halve the sample size a test needs, without changing a single thing about the experiment itself, purely by subtracting out noise you can predict in advance.

CUPED - Controlled-experiment Using Pre-Experiment Data - is a variance-reduction technique that uses each user’s behaviour from before the test to sharpen the estimate of the treatment effect. The short version lives in the CUPED glossary entry; this post is the intuition, the math, and when to reach for it.

The intuition - subtract what you already knew

Imagine you are testing a change and your metric is revenue per user. Users vary wildly in how much they spend. A handful of whales spend hundreds, most spend little, and that spread is enormous noise sitting on top of whatever small lift your change produced. The lift is a whisper in a stadium.

Here is the key observation: a lot of that spread was predictable before the test even started. A user who spent a lot last month will probably spend a lot this month, regardless of your variant. That predictable part is noise you can remove, because it has nothing to do with your treatment.

CUPED does exactly that. For each user it looks at their pre-experiment behaviour, estimates how much of their in-experiment outcome that history explains, and subtracts it out. What remains is the part of the outcome that was genuinely surprising - and it is against that quieter background that your treatment effect becomes visible. The whisper is now in a library.

The math, kept honest

The adjustment is simpler than its reputation. For a metric Y and a pre-experiment covariate X (the same metric measured before the test), CUPED computes an adjusted metric:

Y_cuped = Y - theta * (X - mean(X))

where theta = Cov(Y, X) / Var(X)

The theta term is just the slope of Y on X - how much of Y you can predict from X. Subtracting theta * (X - mean(X)) removes the predictable component while leaving the average unchanged, which is why the estimate stays unbiased.

The payoff is captured by one clean result. The variance of the adjusted metric is reduced by a factor of (1 - rho^2), where rho is the correlation between the pre-experiment covariate and the in-experiment metric. So:

Correlation (rho)	Variance reduction	Rough sample-size saving
0.3	~9%	small
0.5	~25%	noticeable
0.7	~49%	roughly half the traffic
0.9	~81%	a fraction of the traffic

Because required sample size scales with variance, cutting variance in half roughly halves the users you need. That is the whole prize. The one non-negotiable rule: X must come strictly from before the experiment. Use anything from during the test and the treatment leaks into your covariate, and the result becomes biased.

When CUPED helps most, and when it does not

CUPED is powerful precisely when past behaviour predicts future behaviour, and useless when it does not.

• Big wins: returning-user products where engagement, revenue, or session counts are sticky month to month. A user’s history is a strong predictor, rho is high, and the variance reduction is large.
• Little to no benefit: metrics with no meaningful pre-period. First-time-visitor conversion, brand-new signups, or a metric that simply did not exist before the test has no covariate to lean on, so rho is near zero.
• Small or short tests: the overhead of computing the covariate rarely pays off on a tiny experiment, though a good engine applies it for free.

The rule of thumb: the more a user’s past predicts their present, the more CUPED is worth.

Which platforms actually do this

CUPED is not a checkbox on a basic flag tool. It is a feature of experimentation-first platforms with a real stats engine.

GrowthBook includes CUPED in its engine alongside sequential testing, Bayesian analysis and SRM checks. Being warehouse-native, it applies CUPED against your own event history in BigQuery, Snowflake or Databricks, which is exactly where the pre-experiment covariate lives. The MIT core self-hosts free. The trade is a genuine statistics learning curve and a dependency on having an instrumented warehouse.

Statsig was built by ex-Facebook engineers to recreate a rigorous internal experimentation stack, and CUPED and sequential testing are part of that engine. Flag and config checks are unlimited and free on every tier, with analytics events as the meter, so the heavy statistical machinery is cheap to run. Statsig has been owned by OpenAI since September 2025.

CUPED itself was published by Microsoft’s experimentation team and is used at scale by companies like Netflix, which speaks to its pedigree - this is production-grade methodology, not a gimmick.

Where CUPED fits in your testing practice

CUPED does not change how you design a test - you still need a hypothesis, a primary metric, and a sample-size plan. It changes the economics of that plan by shrinking the traffic required to hit the same power. Think of it as free acceleration you turn on in the engine, not a substitute for discipline.

Pair it with the fundamentals: our statistical significance guide explains what you are trying to detect, how to run an A/B test puts the workflow together, and the best experimentation platforms roundup shows which engines carry CUPED and which only claim to do A/B testing. If your tool supports it and your metric has a predictive pre-period, turning on CUPED is one of the cheapest ways to reach a trustworthy answer faster.