...

Even though we work hard to write clear and unambiguous tasks, the results sometimes make it plain that a task was not clearly understood (or was clearly misunderstood) by our participants. Because most tree testing is done unmoderated, participants can’t ask for clarification when they have trouble understanding a task.

•  example of misunderstood task - BNZ IB notifications
  

For example, when we helped a bank run a tree test on their self-service website, we included this task:

How would you get automatically notified when your cheque-account balance goes below $100?

In the results, we saw that:

• Some participants went to the Account Status section to see if there were any alerts.
  
  
• Other participants went to the Preferences section to set up a low-balance alert.
  
  

When we re-read the task, we saw that it could have interpreted as either getting the alert, or setting it up. (We actually meant the latter.)

If we’re doing another round of testing, we should revise these murky tasks so we can get higher-quality results. This makes before-and-after comparisons harder, but comparisons are less important than clear primary findings about the tree.

•  example of revising a task

In the example above, we clarified the task wording so that we could judge the results better:

How would you set up an automatic notification for when your balance goes below $100?

...