...

The big advantage of using online testing tools is that once we launch a study, it runs by itself; people can participate any time (and any place) they want, and we don’t have to be there to moderate the session.

The flip side of this, of course, is that we can’t ask remote participants why they made certain choices or what specifically confused them in a certain task.

...

Each in-person session is run much like a standard usability test:

• We ask the participant to do the tree test (using the testing tool) and encourage them to “think aloud” as they go.

• If they struggle to find a correct answer, we ask neutral questions to find out why they’re having trouble (“Tell me what you’re thinking now…tell me more about that…” and so forth).

• While not every participant will have trouble with the tasks we’re concentrating on, we hope to get a few who do, so we can find the cause of the problem.

For more handy tips on testing in person, see Nick Bowmast’s short article. 

Recording results

Another way to see why participants are making puzzling choices is to employ a remote usability-testing tool that records a screencast of the user's session. (UserTesting.com and TryMyUI.com are popular examples.)

Instead of using this type of tool to test a website, we give them the web address of the tree test, and we get back screen recordings of users doing the study. Because we can instruct them to "think aloud", we'll be able to hear their reasoning (and possible confusion or misunderstanding) as they click through the tasks.

See this UserTesting.com blog post for an example of how recording remote sessions helped the author understand what needed fixing in their tree test.

...

Next: Closing the test