Where they gave up

Sometimes, when participants don’t find an answer they like, they back up and try a different path. Other times, they just give up.

Whether this happens after a few tentative clicks, or after an exhaustive search of every likely path, the fact that participants give up tells us that our tree failed those people for that particular task.

It’s useful, therefore, to look at tasks where an abnormally high percentage of the participants give up (that is, instead of choosing an answer, they click a “Skip” button or something similar). For a tree of average size and complexity, we typically look for skip rates of 10% or more.

Here’s an example from New Zealand’s Ministry for the Environment. They suspected that it was hard for people to find out about consultations (where the government asks the public for input on proposed laws), so they created a task to find out. Here’s the result:

Open

The first thing we see is that they were right – only 25% of participants found the correct answer, because it was buried in an obscure part of the site. Note also the low Directness score (showing that more than half the participants had to back up at least once) and the long time they spent looking (more than 40 seconds on average).

The final evidence that this was really hard for participants is the skip rate – 15% gave up during the task, compared to an average skip rate of 3% for the other tasks.

When we see a high skip rate for a task, we do two things:

• We look at the click paths for this task and see if there are any patterns in where the participants gave up. If they all lost hope in the same section, that helps us narrow down the culprit.

• We compare this task to other tasks with high skip rates:

  • If we see that a certain section has a lot of drop-outs between the tasks, this suggests where we should fix the tree.

  • If we see that certain types of tasks have high skip rates (for example, tasks where we ask them to do research on a product), this suggests that the tree is not supporting this type of task as well as it should.

Next: Confidence