...
(You’ll also need a way of identifying which participants belong to which user group. We often do this with survey questions – see Chapter ~, Setting up a test Adding survey questions in Chapter 8.)
Some user groups are more important than others, and your pool of participants is often limited, so try to get more participants from your major groups. If you end up with too few participants of a less important group, that’s something your project team can probably live with.
...
- Each participant does all tasks.
Suppose you have 10 tasks in your tree test – that is, 10 things that you want your participants to find, and this provides adequate coverage of the important parts of your site tree.
Suppose you decide that each participant should do all 10 tasks. This is a reasonable number because, as we saw in Chapter ~, Writing Tasks How many tasks? in Chapter 7, 10 tasks makes for a quick test and minimizes the learning effect.
Because each participant is doing all the tasks, you would simply aim for 50 participants of each user group. - Each participant does half the tasks.
Suppose now that you actually wrote 20 tasks, perhaps because the tree is large and 10 tasks just wasn’t enough to test everything you wanted to cover.
If you asked each participant to do all 20 tasks, the number-of-participants answer is the same – about 50 per user group.
However, we saw in the Tasks chapter that it’s not a good idea to ask participants to do that many tasks: it takes too long, they get bored or tired, and they are more likely to “learn” the tree (which skews your results).
If you asked each participant to do 10 tasks, that’s half the tasks in the test, so you will need twice the number of participants (about 100) to get each task “hit” by enough people.
...