How to Calculate Sample Size for A/B Testing (2026 Guide)

Sample size is the input that quietly decides whether your A/B test can succeed at all. Run too few users and even a genuine improvement stays buried in the noise, so you call a real winner “no effect” and move on. Calculate it up front and you know before you start whether the test is worth running. This guide shows you exactly how the number is built.

Sample size is the amount of traffic per variation you need to reliably detect an effect of a given size. It is not a fixed number you look up - it is the output of a short calculation driven by four inputs. Get the inputs right and the arithmetic is straightforward.

The four inputs that drive the number

Every sample size calculation, in every tool, comes down to these four levers.

• Baseline conversion rate. Your current rate for the metric you are testing, for example 5 percent of visitors buy. This sets the amount of natural variation in your data.
• Minimum detectable effect (MDE). The smallest improvement you want the test to be able to catch, expressed as an absolute jump (5 percent to 5.5 percent) or a relative lift (a 10 percent improvement). This is the lever teams set wrong most often - see our minimum detectable effect explained guide.
• Significance level (alpha). Your tolerance for a false positive, almost always 5 percent, which corresponds to 95 percent confidence.
• Statistical power. Your chance of detecting a real effect if one exists, conventionally 80 percent. Higher power means a bigger sample.

Change any one of these four and the required sample size moves. Nail them down before you touch a formula.

The formula, in plain terms

For a test comparing two conversion rates, the standard per-variation sample size is:

n = (z_alpha + z_beta)^2 * [ p1(1 - p1) + p2(1 - p2) ] / (p2 - p1)^2

Here p1 is the baseline rate, p2 is the baseline plus your MDE, and the z values come from your significance and power settings. For a two-sided test at 5 percent significance and 80 percent power, z_alpha is 1.96 and z_beta is 0.84, so (z_alpha + z_beta)^2 is about 7.84. This is the classic two-proportion formula and it is what calculators run under the hood.

The shape that matters is the bottom of the fraction - the effect size is squared, so the sample size grows with the inverse square of the effect you want to detect.

A worked example

Say your checkout converts at 5 percent and you want to detect a 10 percent relative lift - moving the rate to 5.5 percent. So p1 is 0.05, p2 is 0.055, and the absolute difference is 0.005.

n = 7.84 * [ 0.05(0.95) + 0.055(0.945) ] / (0.005)^2
  = 7.84 * (0.0475 + 0.051975) / 0.000025
  = 7.84 * 0.099475 / 0.000025
  ~= 31,200 per variation

You need roughly 31,000 users in each arm, about 62,000 total, before this test can reliably tell a 10 percent lift apart from luck. If your checkout sees 5,000 visitors a week, that is a 12-week test - which is a useful thing to know before you commit, not after.

Why the MDE dominates everything

Now watch what happens when you loosen the MDE from a 10 percent relative lift to a 20 percent one (p2 becomes 0.06):

n = 7.84 * [ 0.0475 + 0.06(0.94) ] / (0.01)^2
  = 7.84 * 0.1039 / 0.0001
  ~= 8,150 per variation

MDE (relative)	Absolute diff	Sample per variation
20 percent lift	0.010	~8,150
10 percent lift	0.005	~31,200
5 percent lift	0.0025	~124,000

Halving the effect you want to detect roughly quadruples the traffic you need. That inverse-square relationship is the single most important intuition in experiment design. It is why low-traffic teams should test bold changes with large expected effects, not button-color tweaks, and why demanding to detect tiny lifts commits you to tests that run for months.

Common mistakes that break the math

• Guessing the MDE from hope. Teams pick a tiny MDE because “even a 1 percent lift is worth it,” then are shocked the test needs half a million users. Set the MDE to the smallest lift that is genuinely worth shipping, not the smallest you can imagine.
• Using the wrong baseline. A sitewide conversion average is useless when you are testing one funnel step. Measure the baseline for the exact page and metric under test.
• Ignoring the number of variants. A/B/n tests split traffic across more arms and test more comparisons, so each arm needs its full sample and you should tighten significance to offset multiple comparisons.
• Stopping early when it looks significant. This is peeking, and it silently inflates false positives. Commit to the sample, or use sequential testing. Our statistical significance guide covers why.

Where the tools help

Most experimentation platforms bundle a power calculator so you do not do this by hand, and the better ones fold variance reduction into the estimate.

Statsig ships sequential testing and CUPED variance reduction. CUPED uses pre-experiment data to shrink the noise, which effectively lowers the sample size you need to reach the same power - a real edge when traffic is scarce. GrowthBook carries the deepest open-source engine, including CUPED, sequential testing and sample ratio mismatch checks that confirm your split actually landed at the ratio your sample size assumed. Optimizely built its reputation on a stats engine designed to stay valid under continuous monitoring, so you are less exposed to the peeking trap.

For choosing between them, see the best experimentation platforms roundup and Statsig vs GrowthBook.

The bottom line

Calculate sample size before you run the test, from your real baseline, an honest MDE, 5 percent significance and 80 percent power. The formula is one line, the intuition is inverse-square, and the payoff is knowing in advance whether an experiment can answer your question or is doomed to run forever. Size it first, commit to the number, and let a stats engine handle the rest.