How to create an A/B test?

1. What is a variation test?

2. Before starting the test

3. Getting Started on Creating the Test:

4. Implementing Variation Strategies

5. Running the A/B test

6. Reviewing Test Results / Analytics

7. Ending the Test and Implementation

1. What is a variation test?

First, let's define what a variation test is and why it's valuable.

If you're already familiar with this concept, you can skip this explanation and proceed to the section on how to get started.

A variation test, often referred to as an A/B test or split test, is a controlled experiment where two or more versions (variations) of something, such as a collection page, are presented at different times to different audiences. The primary aim is to determine which variation performs better in terms of achieving specific goals, like higher conversion rates, sales, or engagement.

With the app, you can conduct a variation test for your collections, enabling you to identify which strategies perform best in terms of conversion, sales, and more. These tests are integral to the process of optimizing conversion rates and continuously improving your store’s performance.

2. Before Starting the Test:

• Define Your Testing Objective: First, decide what you want to test. You can experiment with different filters and sorting methods. In this example, we're testing the best sorting strategy for our "White" collection. 

• We encourage you to advise with our success manager to help you set your first goals.

• Identify Variations: Deciding on the number of variations.

Once you start an A/B test 2 variations will be created automatically, but you can add more variations.

In our example, we will have 2 variations: the original collection (sorted with Kimonix's AI smart sorter) and a variation with custom sorters of our choice.

3. Getting Started on Creating the Test:

Option 1: From the A/B Tests Page

• Go to the A/B Tests page

• Click on "Start a new test."

• A pop-up will appear. Ensure you select the correct collection from the dropdown.

• Click on "Create test."

• A test is then created with 2 variations that are identical to the original collection.

• You then have the option to change and set the strategies for each variation.

Option 2: From the Collections Page

• Go to your Collections page.

• Find the collection you want to test.

• Click the three dots on the right side of the collection.

• Choose "Create A/B test."

• A test is then created with 2 variations that are identical to the original collection.

• You then have the option to change and set the strategies for each variation.

Good to know: On your Shopify admin, each variation will be presented as a separate collection.

4. Implementing Variation Strategies:

Now, it's time to set the strategies for each variation.

• Click the 3 dots and then the edit button found to the left of the variation name.

• Add a different variation name for easy identification.

• This is exactly like building a collection where you can set the strategy for each variation.

• Adjust the parameters based on the sorting strategy chosen for each variation.

• REMEMBER to save each variation separately.

• The collection handle will stay the same and have an additional query string parameter, "Kimonix-target". This addition has no effect on your SEO, Google Analytics etc.
• The active variation will automatically change every hour on your store site.
• The products in the tested collection will be sorted according to each of the variation strategies, and the active variation will rotate every hour.

6. Reviewing Test Results / Analytics:

• For each variation, you will get the following metrics: Collection views; Product views; CTR; Sales; Sales quantity; Daily sales; Daily sales quantity; Conversion rate; Appeared in orders, and Gross profit.
• The calculation of each metric can be viewed by clicking the small “i” icon next to each metric.
• It is recommended to leave the test running for at least 7 days in order to view the most accurate results.

• Once you are ready to choose the winner, click on "End test" under the test name.

• A pop-up will appear where you can select the winning variation.

• If you choose one of the variations as a winner, the original collection will be replaced with the winning variation strategy.

• Select the winning variation and click “End test”

Good to know: you can also choose to stay with the "Original collection" and not implement any variation.

Your test is now complete, and the chosen variation strategy will replace the original collection automatically on your Collections page.

That's all for this tutorial! 👏 Until the next one, happy testing and optimizing your collections!

Have any feature requests? Need our help?