A free comprehensive guide for evaluating site structures
As we saw in Different tasks for different user groups in Chapter 7, we may have separate tests set up for separate user groups. We can then use several methods from Chapter 9 - Recruiting participants to direct each participant to the right test.
But what if we have several tests that all target the same users? This is actually quite common, because we often want to test 2 or 3 alternative trees against each other to see which ideas work best.
There are two basic ways of splitting our participants randomly among tests – using mutually exclusive links and using code.
Splitting using mutually exclusive links
The simplest way to split participants randomly is to set up the tests as a list of links, then construct some arbitrary way of getting participant to pick different links.
We usually do this by asking participants to choose the link that corresponds with the first letter of their first name. If we are running 3 tests, for example, the first link is A – H, the second is I – P, and the third is Q – Z:
If we want a weighted split (e.g. 70/30) for some reason, we can adjust the letter ranges accordingly. Midway through the test run, if we notice that we’re not getting enough people on test 3 (for example), we can expand its letter range to help it catch up.
Splitting automatically using code
Here’s how it looks in a web ad’s explanation page:
As a workaround, we could make the email invitation go to a web page on our site that shows a “One moment please…” message, does the random-test-by-code selection, then automatically redirects to the corresponding test address.
Next: Launching the test(s)