A/B Test

Category: Growth

A controlled experiment that compares two variants to see which performs better on a defined metric.

A/B testing works when three conditions hold: enough traffic to reach statistical significance (usually 5,000+ conversions per variant), a clean random split, and a single primary metric. Common pitfalls: peeking early, running too many tests at once with overlapping audiences, and declaring a winner on a directional lift that isn't statistically meaningful.

See also: cro, multivariate, significance