Run an A/B test and publish the winner
An A/B test shows some visitors a different version of your widget, like another headline or button colour, so you can keep the version that gets more responses. This article covers setting up the versions, comparing them, and publishing the winner.
Create test versions
Your original widget is the control. Every other version you add is a variant.
- Open the widget you want to test and create a new test. Give it a name, and a description if you like.
- Add a variant and give it a name that says what's different, such as "Green button" or "Shorter headline".
- Choose how the variant starts:
- Copy from control: starts as an exact copy of your current widget.
- Start from scratch: starts blank.
- Duplicate from variant: starts as a copy of another variant you already made.
- Edit the variant so it differs from the control in the thing you want to test. Change one thing at a time, so you know what caused any difference.
- Set how much traffic each version gets. Each variant has its own traffic share, and the control has one too.
- Start the test. You can pause it at any time.
Check your plan
A/B testing is called "version tests" on the pricing page. How many you can have depends on your plan:
| Plan | Version tests |
|---|---|
| Free | Test two versions |
| Starter | 5 per workspace |
| Pro | 20 per workspace |
| Business | 100 per workspace |
The limit applies across your whole workspace, not to a single widget.
Compare results
Let the test run until enough visitors have seen each version. Then compare the versions side by side:
- Views: each time someone opens a page that shows your widget.
- Responses: each time someone fills in a form, answers a quiz, books or signs up on your widget.
The version with more responses is usually the winner. Because each version may have been shown to a different number of people, compare responses against views, not just the raw response counts. A version with 40 responses from 1,000 views is doing better than one with 45 responses from 2,000 views.
If you want to stop collecting data before you decide, pause the test.
Publish the winner
When you've picked a version, publish it so every visitor sees it.
- Open the test and choose Publish Winner.
- In the Publish Winner dialog, open the Winner menu and select the version you want. The list shows Control (Base Embeddable) followed by each of your variants by name.
- Read the yellow warning. Publishing overwrites your base widget's code with the winner's code.
- Click Publish Winner. The button is greyed out until you select a version. Click Cancel if you want to back out.
What happens when you publish
- The winner's code is copied to your base widget, replacing what was there. This cannot be undone from the dialog, so make sure you have a backup of your current version if you may want it back.
- The test is marked as completed.
If publishing fails, the dialog shows an error message and nothing changes. Try again, and contact support if it keeps failing.
After you publish, your widget shows the winning version to everyone.
Did this answer your question?
Your feedback helps us improve.
