Conversion

what is A/B testing and why your ads and pages need it

Tom Claydon, Co-founder of Nudge1 min read

A/B testing shows two versions of something, an ad, a headline, a page, to similar people at the same time, and lets the results pick the winner. It is the difference between marketing that improves every month and marketing that costs the same forever while everyone argues from opinion.

how a fair test works

Change one thing. Split the audience randomly. Run both versions simultaneously so time and luck cannot take credit. Wait for enough data to mean something, then keep the winner and test the next thing against it. The discipline sounds dull, and the compounding is anything but.

what to test first

Test where the traffic is and where the stakes are: the ad headline people see before anything else, the opening line of the landing page, the words on the button, the offer itself. A stream of small verified wins beats the great redesign nobody can measure, because each win becomes the new floor.

why this is the whole game in paid search

Click prices are set by auction and roughly similar for you and your competitor. What differs is what happens after the click. If your page converts better because it has been tested relentlessly, every click you buy simply produces more than theirs does, at the same price. Testing is how smaller advertisers beat bigger budgets.

the honest catch

Low traffic means slow tests, and a test without enough data is a coin flip wearing a lab coat. If your volumes are small, test bigger differences rather than button colours, run tests longer, and accept that some months the honest reading is not yet significant. Anyone who reports a winner from a handful of clicks is decorating noise.

Common questions

How long should a test run?

Until the result would embarrass someone who bet against it, which depends on your traffic. Full weeks at minimum, so weekday and weekend behaviour both count.

What should we never test?

Trivia while fundamentals are broken. If tracking is wrong or the offer is unwanted, no headline test rescues it. Fix the foundation, then optimise.

Do AI tools replace testing?

They generate variants faster than ever, which makes testing more valuable, not less: more candidates, same need for evidence about which one actually wins.
Portrait of Tom Claydon, Co-founder of Nudge

Tom Claydon

Co-founder of Nudge

Ask Tom anything about search, he answers on WhatsApp.