A/B testing, sometimes referred to bucket testing or split-run testing, is simply an optimization method in which you set up an experiment to see which version of something performs better. It’s one of the best ways to optimize marketing materials such as advertisements and websites.


What Can You Test?

A/B testing necessarily has a marketing focus because its goal is to determine which option appeals to users the most and will thus net you more sales conversions, downloads, or website visits.

It offers the means to test different options of all sorts of things. For example, you can A/B test online ads, advertising copy on your website landing pages, sales funnel Emails, even something as specific as a single headline – with the goal of determining which option performs better for whatever specific goal you have in mind, be it purchases, subscriptions, or clicks. A/B testing can also be used to test things such the appeal of different graphics (including icons, buttons, images used on your website, and even different color schemes), and if you have a beta group available to you, you can also test things such as different user interfaces and potential product changes.

A/B testing gives you the opportunity to see if something will have a beneficial effect before spending considerable money on fully implementing the change, thus preventing costly errors that have to be rolled back (not to mention the potential of upsetting users and customers if a change is not well-received), while allowing you to improve and optimize your user experience over time.

You can even test different video content. has helped clients create different versions of a video for them to A/B test as part of bigger marketing campaign. If you want to test different video versions, you want to consider things such as:

·       taking different approaches to explaining the same content

·       changing the script

·       changing any text in the video

·       different styles of video (e.g. animation versus live-action)

Since video is one of the most effective ways to reach out to your users, optimizing video content through A/B testing is a worthwhile endeavor.


How Does It Work?

A/B testing sounds pretty simple – and it is, as long as you approach it systematically. Here are six basic steps for implementing A/B testing in your business.


Data Analysis

To use A/B testing wisely, your first step is to identify potential areas where improvement is needed. For example:

·       a landing page with a low conversion rate

·       a webpage where visitors are often exiting your website

·       an advertisement that isn’t providing a strong ROI

You can also use A/B testing to make further improvements on something that’s already working well for you. You might have a product page that’s already doing great, but your data shows you are still losing a certain percentage of potential sales.



Be sure you set a goal for your testing. A goal can be anything from increasing the number of clicks on an ad to getting more customers to sign up for your email newsletter. You might decide that for a change to be worth implementing, it would need to increase sales conversions by 5%.


Testing hypothesis

With a clear goal on what you want to get out of testing in mind, you can then work on brainstorming ideas for improvement. This is where you should feel free to get creative, coming up with a list of ideas that you can then winnow down.

As you make your list, jot down why you think an option might perform better, and don’t forget to consider the difficulty and expense of implementation – that doesn’t mean you should automatically eliminate more expensive or difficult ideas, but it’s definitely something you want to have thought about before you begin testing.

When narrowing down your list of options you want to test, focus on the “low-hanging fruit” first. These are the ideas that:

1.     are simple to test, meaning they don’t require a great deal of time and effort to create

2.     have strong reasons for why they will perform better

3.     are also simple to implement once the testing is done if they’ve proven effective


Variation Creation

Once you’ve winnowed the list down, you need to actually create the variations you’re going to try. Do you want to try different color schemes on a landing page? Or a different opening line or subject line in a sales email? Note that there are numerous A/B testing tools out there, and many of them have visual editors that can help you create variations. If you’re creating different webpage versions or anything which involves clicking on a button or performing a specific action, don’t forget to test your options to make sure they are functional before the test begins. Make sure that the new button really does go to the landing page, and that there are no typos or grammar mistakes in any text you changed on the new page.



Don’t try testing multiple different variations at once, even if you’ve come up with a number of possibilities; there’s a reason it’s A/B testing and not A-Z testing. Testing two options at a time is best. If you have more ideas than that, test two, and then run subsequent tests, keeping the top performer from the prior test to run against one newcomer.


Data Analysis Again

Once you’ve run the test, you have to look at the results to determine whether or not your stated goals were met. Sometimes, after analyzing the data, you may decide to implement the change even if your goal wasn’t met; perhaps it was only off by a fraction, or over the course of the test, your goals may have changed. It’s also possible that after analysis, you discover something else or another possibility you want to test, in which case the process begins again – in fact, good A/B testing is something that can and should be done on a regular basis as you gather more data and learn more about your users and what appeals to them.


A/B testing is a great way to take the guesswork out of your marketing and user experience. Don’t know which landing page is best? Test them both! Not sure which ad copy or sales email will give you the best results? Test! Using A/B testing will, over time, lead to steady improvement of your advertising and marketing strategies.