This article is the companion piece to our first article on A/B testing, and is the result of many clients asking us how to properly test the video ads we’ve created for them.
Why A/B Test?
Quite simply, A/B testing takes the guesswork out of your marketing and user experience. It gives you the opportunity to see which version of a video does best in terms of engagement, conversions, and sales before you commit valuable marketing dollars to implementing it on a large scale. A/B testing literally tests your users’ behavior, allowing you to discover which options work best to get them to do what you need them to do – whether that’s downloading your app, signing up for your newsletter, or visiting your website.
A quick note: A/B testing isn’t something that happens fast (though you’ll definitely start to see results and data that you can act on fairly quickly), or that you do once and forget. Smart marketers are constantly testing and then tweaking their videos (and other materials) to continuously improve them. Used over time, A/B testing will absolutely result in steady, lasting improvements to your marketing and advertising.
How To A/B Test Videos
Okay, you’ve got your two videos, and you want to run an A/B test to see which one performs better. The most important thing to remember about this “test” is that it’s not an internal test that no one will see – it’s live, so the videos will be seen by potential customers and users. This means you should treat it as seriously as any other marketing campaign (it just has a lower budget).
To start, you’ll create two ad campaigns on whatever social media network you’re wanting to do the test on. All of the variables should be exactly the same – identical text, identical target audience demographics, and identical time period. The only difference between the two should be the video. This is absolutely essential, as you need any difference in performance to be due to the video, and not to anything else that was different about the ad. The easiest way to ensure this is to create the first ad campaign, get everything finalized, and save it. Then, let it sit for a day to allow yourself to come back and give it a “fresh” look. After that, go over it one more time to make sure there are no other tweaks you want to make. Once you’re sure it’s finished and ready, duplicate it, then switch out the first video with the second and leave everything else as-is.
You should now have two draft ads ready to go. To conduct the A/B testing properly, they should both be launched at the same time so that they run simultaneously, and both should run for the same amount of time. Ideally, this should be a week at minimum (2-3 weeks is best) to give you time to gather enough data to really see the differences in performance. You’ll be able to keep an eye on the results during the test, and at the end, you should have a clear idea of which video netted you the best results.
What Should You Analyze?
Some of what you need to analyze may be video-dependent and thus difficult for us to speak to (for example, if you have a call-to-action embedded in one video and not the other, then one of the data points you want to analyze is whether or not viewers actually engaged with that call-to-action), but in general, when you run an A/B test, you’re looking at the following:
- Which ad had the most views, i.e., was seen by the most people? Also, if the platform gives you data about how much of the video was watched, which video had the most fully completed views? These might seem like the most important metrics, but while it’s good to take these numbers into account, the ad seen by the most eyes isn’t always the ad you want to keep. Instead, consider the next two bullet points.
- Which ad had the most viewer engagement (not just views, but likes, reactions, comments, and shares)? In particular, the engagement you want to focus on is: how many people actually clicked on and watched each of the videos you’re testing?
- Which ad had the most conversions (if applicable – for example, if the goal was to get the viewer to click on a link to your website, or downloaded your app)? In campaigns where conversions are the goal, this is the most important metric.
When you’re looking at your test data, you want to focus on the data points and the numbers that matter most – and that depends on what your goal was. If your goal is, “I want to know whether video A or video B generates the most views,” then you’re going to focus on that first bullet point. But, if your goal is, “I want to know which video gives us the most conversions,” then the third bullet point is your sweet spot. And sometimes, the results are going to surprise you – for example, you may find one video gets more views, but it’s the video with fewer views that actually generates more conversions.
Both Facebook and Twitter have information and instructions on how to perform A/B testing via campaign creation right inside their own publishing tools, making it even easier for everyone to do their own social media A/B testing.
Two Final Notes
First, A/B testing results can be surprising; the winning option is often the surprising or unintuitive one, so be careful not to second-guess the test. This is one time not to “go with your gut” against the test results. Trust your test, then put your budget behind the winning option as your ad!
And second, A/B testing isn’t “one and done.” In order to get the most use out of A/B testing, you need to do it again and again. Analyze the data, implement your changes – and then test your next set of possibilities. A/B testing is an iterative cycle, wherein you are constantly making improvements based on the last test results while coming up with new things to test. Sometimes this means making a variation of your more successful video and testing it against the existing version, and sometimes it’s creating two entirely new variations to pit against each other.
A/B Testing isn’t complicated or difficult, as long as you remember the essentials: create your ads, publish them simultaneously, and gather/analyze the results. Making A/B testing a natural part of your marketing strategy will pay dividends, especially in the long term. If you’re in need of videos for A/B tests, please reach out. Our company has years of experience in creating multiple versions of demo videos and marketing videos perfect for testing purposes.