A/B testing (or split testing) your prototypes with Maze

Split testing, also known as A/B testing, is a method of testing two design variations to see which one performs better.

You can perform split testing with your prototype using Maze. With split testing, the aim is to test a single variable to learn how it affects overall performance. To get the most out of split testing, establish the metric you'll be measuring before you test. For instance, you could improve the click rate of a call-to-action (CTA) by testing two variations of the CTA button.

In this article:

• What you can validate with A/B testing
• Best practices
• Known limitations
• Using the Duplicate feature to A/B test your prototypes
• Changing the order of blocks
• Comparing reports
• Additional resources

What can I validate with A/B testing?

• Text: titles, descriptions, or CTAs
• Visual elements: images, icons, colors
• User flow: the path taken by the user to move through a prototype

Best practices

• When creating different versions for testing, we recommend changing only a single variable, so that the results accurately reflect that variation and not other factors.
• For the best results, randomize the selection of testers for each test version.

Current limitations

• We don't currently support creating A/B test variants for a single maze — creating an A/B test would require you to use the Duplicate feature, as seen below, and circulate two separate maze URLs.
• Because split testing requires creating two separate mazes, we don't currently have a way to randomize which version your testers will see, or to prevent testers from taking both tests. As a workaround, while we don't directly support or endorse them, there are several third-party custom URL tools you might use to generate and distribute your maze share links — for instance, Linkly or Nimble Links.
• If using hired testers and placing panel orders for each maze, you may get the same testers in both mazes. This is especially true if you make both purchases back-to-back and/or target a more niche audience.
• Currently, it's not possible to add two variations to the same Reach campaign. To A/B test with Reach, you must create different campaigns with two different segments.

Using the Duplicate feature to A/B test your prototypes

You can run split testing in Maze using the Duplicate feature. Here's how you do it:

1. Create a new draft maze for Version A, and set up missions and questions for your participants to complete.
2. When you're ready, send Version A live and start testing.
3. Open your prototyping tool and create a new version of your design (e.g., change the color or the copy of the CTA button). You can either make changes to the existing prototype, or create a new prototype. Save all the changes.
4. Back in Maze, duplicate the original maze you created in step 1. This will create a new maze with the same missions and questions as in version A. This new maze will also include the latest design changes.
5. Open the duplicate maze — this will be your Version B. Here, you have two options:
   • Use the same prototype
   • Unlink the prototype and import a new one
6. When you're ready, send Version B live and start testing.
7. When testing is done, compare the results and implement the winning design.

Option 1: Use the same prototype

If you change the original prototype and then duplicate the maze, the new maze will also include the latest design changes. This is a good option when you want to use a different version of the same prototype you previously imported.

This option allows you to preserve the mission paths you previously set.

You can also refresh the prototype in Version B to get the latest changes you've made to the prototype.

Option 2: Unlink prototype

If you want to use a different prototype in your version B, you can unlink the current prototype after duplicating. This will allow you to add a new prototype to your maze. To learn more about adding and unlinking prototypes, check out this article: Mission blocks

This option allows you to pick a different prototype for version B.

However, please keep in mind that you will need to redo all paths across all mission blocks, since unlinking a prototype also irreversibly removes all previously defined paths.

Changing the order of blocks

While Maze doesn't currently support randomizing the order of the blocks in a test, as a workaround, you can edit the maze later on to reorder the blocks. Here's how:

1. Create your maze and send it live. This will be your version A.
2. After you've had enough testers on this version, edit the maze to change the order of the blocks. This will be your version B.

Comparing reports

Once you get both versions tested, you can embed both reports side-to-side in your tool of choice to compare the results easily.

Additional resources

• For additional tips, read our blog post: How to A/B test your prototype with Maze
• See this case study to learn how the team at Braze validated their designs with an A/B test using Maze.