Table of Contents

A/B Split testing

What is A/B Split testing?

A/B Split testing is a method used to compare two different versions of an ad or landing page to determine which one performs better in achieving specific goals.

You can also test the features of a website to find out which improves the user experience (UX). A/B tests help to improve conversion rate optimization (CRO).


A/B Split testing involves creating two variations of an ad or landing page, where each version differs in one or more elements (such as headline, image, call-to-action, or layout).

These are then randomly shown to a target audience, and their performance is measured based on predefined metrics (e.g., click-through rate, conversion rate). The purpose is to identify the most effective version and use it as the basis for future campaigns or optimizations.

Key Takeaways

  • A/B Split testing helps marketers make data-driven decisions and optimize their ads or landing pages for better performance.
  • It is essential to test one variable at a time to isolate the impact of that particular element on the results.
  • Consider statistical significance to ensure the results are reliable and not due to random chance.

How does it work?

Even though each test is unique to the business and goals you want to reach, there is a basic structure you can follow for A/B Split testing.

  1. Define goals that clearly outline the objectives and metrics you want to measure during the test.
  2. Determine the elements or variables you'll change between the two versions.
  3. Create variations: Develop two versions (A and B) of the ad or landing page, with only one variable changed between them.
  4. Split the website traffic by randomly dividing the target audience into two groups. Then you can show each group one of the versions.
  5. Collect data on the performance of both versions based on the predefined metrics.
  6. Compare the performance metrics of each version to determine which one outperforms the other.
  7. Use the better-performing version (the winner) as the benchmark for future campaigns or optimizations.


A store owner running a Google Ads campaign promoting his online clothing store may want to test whether a particular headline works better.

He decides to test two different headlines for the ad: “Shop the Latest Fashion Trends” (Version A) and “Get 50% Off on All Clothing” (Version B). He can track and compare both versions' click-through and conversion rates by splitting his audience.

If Version B generates higher click-through rates and conversions, he can conclude that the discounted offer is more effective in driving user engagement.

This is a simplified example of how the A/B test would work, but there are other metrics and types of tests that are useful for improved UX and CRO.


A/B Split testing typically involves statistical calculations to determine the statistical significance of the results. Various formulas and tools help to calculate statistical significance, such as the chi-square test, t-test, or online calculators designed explicitly for A/B testing.

Online calculators designed specifically for A/B testing can make it easy to calculate statistical significance without having to know how to use the formulas yourself.

It is important to note that statistical significance does not necessarily mean that the difference in performance is meaningful.

For example, a difference in conversion rates of 0.1% may be statistically significant, but it may not be meaningful enough to warrant making a change to the web page or other marketing asset.


Should I always implement the winning variation?

While the winning variation typically performs better, you need to consider the context and your specific goals. Sometimes, a slightly worse variation on one metric may have other benefits, making it more valuable for your overall marketing strategy.

What metrics should I consider during A/B Split testing?

The choice of metrics depends on your goals. Typical metrics include click-through rate (CTR), conversion rate (CVR), bounce rate, average session duration, and revenue per visitor (RPV).

How long should an A/B Split test run?

The test duration depends on factors such as traffic volume, conversion rates, and the desired level of statistical significance. Ideally, you should run tests until you reach statistical significance or until you have enough data for meaningful analysis.

Related Terms


What is remarketing? Remarketing, also called retargeting, is a digital marketing approach that engages audiences who previously interacted with a brand to encourage specific actions.

Ad Variations

What are ad variations? Ad variations are different versions of your ads that you create to test and optimize your campaigns, which can help you

Ad Scripts

What are ad scripts? Ad scripts are snippets of code written in JavaScript that allow you to automate and customize various aspects of your Google

Activity rate by channel

What is ‘Activity rate by channel’? Activity rate by channel measures how often users engage with your ads across different advertising channels. This metric considers actions


What is Abandonment-rate? Abondment-rate is the percentage of users who click on a Google Ads ad but then leave the landing page without completing the

Brand-Focused Shopping Campaign

What are Brand-Focused Shopping Campaigns? Brand-Focused Shopping Campaigns are a type of Google Shopping campaign that targets users who are searching for your brand or

Want to hire us to grow your ecommerce store?

We’ve generated millions for our clients using the methods we’ve developed over the years with a singular focus on Google Ads and CRO for ecommerce. To discover how we can help grow your business:

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors

Subscribe to Our Newsletter to Improve Your eCommerce Digital Marketing!

Thank you for your interest in our newsletters. We will send you only the most important news and updates. No spam, we promise.

We are committed to protecting your privacy, as described in our Privacy policy.

What’s your goal today?

1. Hire us to grow your ecommerce store

We’ve generated millions for our clients, using the methods we’ve developed over years with a singular focus on Google Ads for ecommerce. To discover how we can help grow your business:

Claim your FREE website strategy session with one of our renowned experts.

2. Learn & develop your ecommerce skills

Watch our highly regarded YouTube channel, followed by over 23,000 store owners and ecommerce learners from around the world.

You can also subscribe to our weekly email newsletter to get real-world tested growth and optimization strategies, as well as notifications on anything you may need to know as an ecommerce store owner.

Browse our blog, written and structured to be the best resource in the world for anyone starting and growing their online store. It includes strategy guides, setup instructions, checklists, and tools.

Explore our KeyCompendium, which is a comprehensive wiki of digital marketing terms. Many readers have commented on the usefulness of this content.

3. Join our team

If you want to join our team—or discover why our team members love working with KeyCommerce—then see our “Careers” page.

4. Contact us

We help ecommerce store owners around the world, so get in touch!