As we saw in Different tasks for different user groups in Chapter 7, we may have separate tests set up for separate user groups. We can then use several methods from Chapter 9 - Recruiting participants to direct each participant to the right test.

But what if we have several tests that all target the same users? This is actually quite common, because we often want to test 2 or 3 alternative trees against each other to see which ideas work best.

There are two basic ways of splitting our participants randomly among tests – using mutually exclusive links and using code.

Splitting using mutually exclusive links

The simplest way to split participants randomly is to set up the tests as a list of links, then construct some arbitrary way of getting participant to pick different links.

We usually do this by asking participants to choose the link that corresponds with the first letter of their first name. If we are running 3 tests, for example, the first link is A – H, the second is I – P, and the third is Q – Z:

If we want a weighted split (e.g. 70/30) for some reason, we can adjust the letter ranges accordingly. Midway through the test run, if we notice that we’re not getting enough people on test 3 (for example), we can expand its letter range to help it catch up.

Beyond being easy to set up, this method is also guaranteed to work in all browsers (for web-ad explanation pages) and all email clients (for email invitations), because it does not rely on JavaScript (see below).

Splitting automatically using code

A more elegant way to split participants randomly is to use a bit of JavaScript code. We present a single link to all participants, but when the link is clicked, it triggers some JavaScript that picks a random number (e.g. 1, 2, or 3 if we’re running 3 tests), then directs the user to the corresponding test address automatically.

Here’s how it looks in a web ad’s explanation page:

Here is an vanilla HTML page with the JavaScript code included. Feel free to customize this as needed:

<!DOCTYPE html>
<head>
  <script>
    <!--
    var links = new Array()

    links[0] = "http://google.com"
    links[1] = "http://bing.com"
    links[2] = "http://duckduckgo.com"

    function getRandomLink() {
      window.location = links[Math.floor(Math.random() * links.length)]
    }
    //-->
  </script>
</head>

<body>
  <a href="javascript:getRandomLink()">Do the study now</a>
</body>
</html>

This works well in an explanation page on the website (that the web ad would direct to), but it may not work reliably in email invitations because some email clients may block JavaScript in incoming messages.

As a workaround, we could make the email invitation go to a web page on our site that shows a “One moment please…” message, does the random-test-by-code selection, then automatically redirects to the corresponding test address.

Next: Launching the test(s)

Tree Testing for Websites

Splitting users randomly among tests

Splitting using mutually exclusive links

Splitting automatically using code