| Table of Contents | ||
|---|---|---|
|
...
You should randomize tasks
Most tree-testing tools give us the option of randomizing the order of tasks that are shown to the participants.
If we don't randomize tasks, each participant will see the same tasks in the same order:
Participant 1 | Participant 2 | Participant 3 |
|---|---|---|
Task 1 | Task 1 | Task 1 |
Task 2 | Task 2 | Task 2 |
Task 3 | Task 3 | Task 3 |
If we do randomize tasks, each participant will get the tasks in a different (random) order:
Participant 1 | Participant 2 | Participant 3 |
|---|---|---|
Task 1 | Task 2 | Task 3 |
Task 2 | Task 3 | Task 1 |
Task 3 | Task 1 | Task 2 |
| Tip |
|---|
For most studies, we should randomize the order of tasks. |
Reasons for randomizing tasks
Why randomize tasks? Because we want to reduce the learning effect on our results:
If we ask all participants to do the tasks in the same order, then they are likely to do better on tasks presented late in the test because they have learned at least some of the tree by then (by browsing it repeatedly). This gives late tasks an unfair advantage.
By putting the tasks in a random order per participant, we make sure that task 15 (for example) is sometimes shown late, sometimes shown early, and sometimes shown in the middle. Mixing up the task order negates ensures that no task gets an advantage by always appearing late in the test.
Not randomizing the first task
Some online tools provide an option to not randomize the first task; that is, the first task in our list is always shown first to participants, even if the other tasks are randomized.
...
What makes a good first "training" task?
The task is not too hard.
We don't want participants to be intimidated or dismayed at how hard our warm-up task is.The most likely paths go at least 2 clicks down the tree.
We want participants to see that the tree is multi-leveled, and is only shown one branch at a time.The results for this task are not important to our study.
Because this is a "throw-away" task, we want to pick something that we don't really care about (because we'll be ignoring this task's results later).
...