Term · Marketing & analytics
A/B testing
A/B testing — also called split testing — shows two versions of something (a page, a form, an ad, an email) to comparable groups of users at the same time, and measures which version produces better results on a chosen metric.
[ 01 ]
Explained
How it works
- Form a hypothesis: "Showing delivery cost on the product page will increase purchases."
- Create version A (current) and version B (changed).
- Split traffic randomly and consistently between them.
- Measure the primary metric — purchases, enquiries — for both.
- When enough data is collected, check whether the difference is statistically reliable.
- Implement the winner, or conclude that the change does not matter.
Sample size and duration
Small differences require large samples. Calculate the needed sample size before starting, from your current conversion rate and the smallest effect worth detecting. Run tests for full weeks to cover weekday and weekend behaviour.
Common mistakes
- Stopping early when one version looks ahead — early leads often disappear.
- Testing many things at once without enough traffic.
- Choosing the metric after the test — pick it beforehand.
- Ignoring segments: a change can help mobile users and hurt desktop users.
- Testing trivial changes on low-traffic pages where no result can be reliable.
When traffic is low
If a page gets little traffic, formal A/B tests may take too long. Qualitative research, best practices and before-and-after comparisons are more practical. See A/B testing and conversion optimisation.
[ 02 ]
FAQ
Frequently asked questions
What should we test first?
Changes with the biggest expected impact on pages with the most traffic and the most drop-off — often headlines, offers, forms and checkout.
Can ad platforms run A/B tests?
Yes. Google Ads and Meta have built-in experiment tools for ads, audiences and landing pages.