A/B Test Calculator
Did variant B really beat A, or is it noise? Enter visitors and conversions to get significance, a confidence interval for the difference, and the uplift.
Control (A)
Variant (B)
Results update as you type.
Method
The calculator runs a two-proportion z-test. The test statistic uses the pooled conversion rate p̂ = (cA+cB)/(nA+nB) and standard error √(p̂(1−p̂)(1/nA+1/nB)). The confidence interval for the absolute difference uses the unpooled standard error √(pA(1−pA)/nA + pB(1−pB)/nB). The normal approximation is reliable when each group has at least about 10 conversions and 10 non-conversions.
Sample ratio mismatch (SRM)
If you intended a 50/50 split but traffic is noticeably uneven, something in the assignment or tracking may be broken, and the result can't be trusted. We run a chi-square test on visitor counts and warn when p < 0.01.
Read results responsibly
- Decide the sample size before the test with the Sample Size Calculator, and don't stop early just because p dips below 0.05.
- A significant result tells you the effect is unlikely to be zero, not that it's large. Check the confidence interval.
- Run tests across full weekly cycles to avoid day-of-week bias.
Full explanation: A/B testing, explained properly.