How to create an A/B test

A step by step guide to setting up an A/B test on your Bundles, Upsells and Quantity Breaks, reading the results and picking a winner.

Run an A/B test on your offers

A/B testing lets you compare two or more versions of a Bundle, Upsell or Quantity Break on your live store, and let real orders decide which one earns more. This guide walks you through setting one up, reading the results, and picking a winner.

Info

A/B testing is currently in Beta and available on request. If it is not switched on for your store yet, reach out to our support and we will enable it for you.

Create your test

The A/B test setup: variants, traffic split and goal

Start a test on an offer

Open the Bundle, Upsell or Quantity Break you want to test and start an A/B test. Your current offer becomes variant A.

Add a variant and change one thing

Add a variant B (and up to two more). In each variant you can change prices, discounts, titles, badges or the quantity tiers. Keep each test focused on one clear idea, so you know what made the difference.

Set the traffic split

Decide how your visitors are divided between the variants, for example 50% / 50%. Use "Even split" for an equal share. Each visitor stays on the same variant for the whole test, so the comparison is fair.

Choose your goal

Pick the metric the test should optimise for: Revenue, Revenue per visitor, Conversion rate or Sales. The goal decides which variant is marked as the winner.

Set the test live

Review the summary and set the test live. While it runs, the offer itself is locked so the numbers stay clean.

Why this is useful:

You decide based on real orders from your own store, not on guesswork
Changing only one thing per test tells you exactly what worked

Read the results

The A/B results panel comparing variant A and B

While the test runs, the results panel compares every variant side by side: visitors, conversion rate, sold items, revenue and revenue per visitor. The strongest variant is marked as the top performer, with the uplift against your control (variant A).

You can filter the results by date range and by variant, and you can pause or resume the test at any time without losing data.

Info

Let a test run long enough to gather a meaningful number of visitors and orders before you trust the result. A difference from only a handful of orders can be down to chance.

Pick a winner

When you are confident, use "Set as winner" on the variant you want to keep. Sectionheroes keeps that variant live and archives the test in your history, so you always have a record of what you learned.

About the numbers

Visitors are counted only after they consent to analytics, which we read from Shopify's Customer Privacy API. Without consent nothing is stored and the visitor simply sees variant A. Revenue is shown net (after discounts, without shipping or tax), pulled directly from your Shopify orders.

See what's new in this release: A/B testing for your offers

Was this article helpful?