A/B test significance calculator
Check whether the difference between two versions is bigger than chance would usually produce. Nothing you enter is saved.
Result
Significant at 95%
- Version A rate
- 2%
- Version B rate
- 2.7%
- Relative change (B vs A)
- +35%
- p-value (two-sided)
- 0.0209
- z-score
- 2.31
How it works
- Pooled two-proportion z-test: z = (rate B − rate A) ÷ √(pooled rate × (1 − pooled rate) × (1/nA + 1/nB))
- Two-sided p-value from the normal distribution; “significant at 95%” means p < 0.05.
How to read it
Decide your sample size before the test and only check the result once it’s reached — stopping as soon as a test looks significant makes the p-value meaningless. A significant result isn’t automatically a large or valuable one.
The full explanation is in our guide: A/B Testing for Higher CTR: How to Run a Fair Test.