A/B Test Significance Calculator

How it is calculated

The tool runs a two-proportion z-test. Conversion rates are pA = cA / nA and pB = cB / nB, where n is visitors and c is conversions in each variant. The pooled rate is p = (cA + cB) / (nA + nB), the standard error is SE = √(p(1 − p)(1/nA + 1/nB)) and z = (pB − pA) / SE. The uplift shown is (pB − pA) / pA as a percentage.

The verdict is fixed: |z| ≥ 1.96, a two-sided 95% level, is reported as statistically significant, anything smaller as not significant. No p-value is displayed. Conversions cannot exceed visitors and visitor counts must be above zero. The test assumes the visitors were split randomly and the sample sizes were chosen in advance.

Common mistakes

• Entering conversion rates in the conversions field. It needs counts (250), not percentages (5).
• Stopping the test the moment the result turns significant. Checking repeatedly and stopping early inflates false positives; decide the sample size beforehand.
• Reading the 20.00% uplift as percentage points. It is relative: 5% to 6% is a 1 point gain, which is a 20% relative lift.
• Treating a significant result as proof the change is large or valuable. It only says the gap is unlikely to be noise at this threshold.
• Pasting in totals from unequal or non-random traffic splits, or counting returning visitors twice, which breaks the independence the test assumes.