What is AB testing?
AB testing is a marketing technique that involves showing two variants of the same digital asset to different segments of the audience at random, all at the same time. Statistical analysis is used to determine which variation performs better for a given conversion goal. Typically, digital marketers use this technique to compare conversion rates before deploying a change to this digital asset. The asset can be a landing page, a product page, a social media post, a newsletter, the checkout page, only to name a few.
Most businesses want to optimise their conversion funnels to increase the chance of visitors converting into customers. AB testing is a way to optimise a website’s conversion funnel. This technique also provides online businesses and eCommerce websites with enough hard data to understand how their audience and potential customers respond to specific elements on their website. eCommerce businesses can benefit a great deal from AB testing when aiming at conversion rate optimisation. In other words, AB testing allows online businesses to get data-driven decisions while making the most out of their existing traffic.
Difference between AB testing and Split testing
AB testing and split testing are two terms which tend to be used interchangeably. However, they are two separate methods of testing different variables.
In split testing, a control version of an asset is compared to a completely different version of it. The version that showcases the best results, e.g conversions, purchases, sign-ups, clicks etc, is the winning one. More on split testing in the next section.
AB testing refers to comparing two versions of the same asset, where only one variable changes for the comparison. For example, when performing AB testing on a landing page, you can only change a headline, the copy, one design element, the CTA button etc. With AB testing, a business can be certain as to which change drove consumer’s behaviour.
What is split testing
Split testing is a testing method that compares two completely different versions of the same asset. Unlike AB testing, where only one variable changes from version A to version B, in split testing you can compare two, three or more, completely different versions of an asset. For example, if an eCommerce business runs a split test on a product page, the variables can be two or more, with totally different pages, headlines, copywriting, product images, CTA buttons, or even prices.
According to the goal of the page, which for the sake of the example will be conversions, the performance can be calculated using KPIs such as conversion rate, bounce rate, add to wish-list etc. The traffic involved in a split test experiment can be all the visitors that have access to that test page or a segment of them based on location, source, device, demographics etc.
AB testing tools
AB testing tools are accessible to eCommerce businesses of all sizes, thanks to a variety of SaaS platforms offering AB testing and split testing tools. If you’re on a lookout for AB testing tools, make sure that the platform you choose offers you the options below:
- Record user sessions
- Capture heatmaps
- Have a user-friendly visual editor to create experiment variations
- Personalisation capabilities for different user segments
Which elements can you AB test?
For online businesses, especially for eCommerce merchants, each and every piece of content is actionable and drives users down your marketing funnel. It’s extremely important that all elements are optimised to their maximum potential, leading to conversion rate optimisation.
Email AB testing
Using email marketing tools, an eCommerce business can test almost any aspect of an email campaign to compare results. In email AB testing, the variables can include subject lines, copy, visuals and call-to-action buttons.
Landing page AB testing
For an eCommerce store, the landing page is the A and Z from a conversion perspective. When running landing page AB tests, you can variate elements such as the layout, offers, the hero image, the headline, all copy, call-to-action-buttons, product images and descriptions.
Copy AB testing
Online users tend to have a decreased attention span, which means that great copywriting should clearly communicate your product messages and CTA’s. Elements that you can tweak for copy AB testing include copy length, tone of voice and word order.
AB testing UX design
UX in eCommerce is particularly important as it should provide the best possible experience for the user, in order to achieve conversions. UX includes all elements, from copy to visuals and CTA buttons, to name a few. Before you start running AB tests on UX, ensure you apply a heat map to see where people are clicking the most on the said landing page. After this point, you can address dead clicks and low conversions by testing different variables until you find what drives the most conversions.
CTA buttons AB testing
Call-to-action buttons are one of the most significant parts of an eCommerce landing page. A CTA AB testing can be applied to various aspects, such as background and font colour, size, copy, word order and position on the page.
Checkout page AB testing
Checkout pages are the make or break step in the customer journey. There are several elements an eCommerce business can AB test on their checkout page. For instance, you can test displaying all payment methods in colour or black and white. You can also try an omnipresent shopping cart footer with a "Pay now" CTA button. Another test would be adding an option to choose the same as the shipping address for the billing details.
How long should your A/B tests run for?
For most AB tests, the duration is not a crucial factor. Statistical significance reflects your risk tolerance and confidence level. For example, if you run an AB test with a significance level of 95%, this means that if you determine a winner version, you can be 95% confident that the generated results are real and not a result due to random chance. It also means that there is a 5% chance that you could be wrong. With that being said, A/B testing for several weeks will give you good results as long as your eCommerce store has consistent traffic. Ideally, consider running AB tests for a minimum of two basic business cycles. This allows you to spot weekly trends and makes your conclusions more efficient.
What are the challenges of AB testing?
AB testing has great benefits for eCommerce businesses, but it also comes with certain challenges. Some common issues big or small companies may face when AB testing include the following.
- Identify where to run AB tests
Businesses should analyse their marketing funnels to identify where customers are dropping out. Is it after the entrance to your website? Is it one step before the conversion? Identify the area that is keeping the customers from taking the next step towards conversion; this area is where you should run your AB test on for more impactful conclusions.
- Run valid tests
For AB tests to drive conclusions for user behaviour, you need to ensure that all experiment parameters are properly set up and monitored in order to stir away from invalidities. For instance, such validity threats may occur when running an AB test during Christmas time when demand is historically high, or when the sample size is too small to determine a valid test.
- AB test result analysis
Analysing AB tests is a step confined at the end of the experiment. However, the key to interpreting AB test results is to set up the hypothesis with the end goal to understand the customer journey and the drop-off points in your funnel. With that being said, the challenge is to prioritise how you can better serve your customer over simply increasing KPI’s.
How to analyse AB testing data
Most AB testing software platforms usually generate reports that show variation results and goals achieved. Based on these results and the most important goal, you can declare a winner when there is a 95% or higher likelihood of winning (a few sections back, we talked about statistical significance).
Conclusion
The competition in eCommerce is fierce and nobody wants to stay behind. AB testing your landing page or checkout page is a competitive advantage for your business.
You don’t need to overcomplicate your life with AB testing. Start by creating the first version of a landing page. Then tweak one element, such as your headline or CTA, to create a variation, then pit them against one another.
Whether or not your AB tests reveal significant differences or not, you can rely on the data for your decision-making process.
Want to find out more about how to accept online payments for your eCommerce business? Drop our payment experts a line.