How to create a split test for your chatbot flow

A common technique when testing a product or a communication approach is to create split tests. Split tests are basically variation of the same content that are randomly submitted to your users. Usually content is split into two tests (A, B tests) but sometimes it can become more complex and even include weighted paths.

This approach is widely used in chatbots too and Xenioo has an incredibly flexible answer to this: the Random Split Action.

The Random Split action will basically trigger its operations based on a given probability chance. It is all very simple and you can start by selecting Random Split from the actions dialog, under the Flow section. After adding the action and completing your flow, you will end up with something looking like this:

Basically this flow says: “50% of the times, redirect the flow to path A instead of going to path B”. With just an action, we’ve already built our A-B test, but we can do (a lot) more than this.

You can see selecting the action that you can actually change the chance associated to the trigger. So for example to have a 20% A and 80% B you would go like this:

Since our Random Split is just an action nothing is stopping us from adding more. We can do A-B-C-n testing for example of weighted branching where just a smaller percent of users will go in another route. Multiple sub-sequential branching is also perfectly supported.

Just remember to keep an eye on the total chance of each single interaction: random splits exceeding 100% will not be executed!

Xenioo: How to create a split test for your chatbot flow