Passing along participant feedback

The main value we get from tree-test participants is the data we collect as they try to complete the tasks we’ve set. That helps us see where our trees are succeeding and failing, and how we can improve them.

A secondary benefit is that we’re getting real users to engage with us, and this is a great opportunity to get a bit of extra feedback from them. Normally we do this by adding a few survey questions to the study.

While some of these questions may be aimed at helping us analyze the results by group (e.g. asking their age, then checking to see if this affects their success rates), some questions may simply be there for gathering some knowledge about our audience. For example, we might have asked which region they live in, which may not make much difference to the tree test itself, but which may be interesting to the Marketing folks.

The most common kind of item to include in online studies like this is an open-ended catch-all question such as “Did you have any other feedback about our website?”, where the user can type away with questions, comments, suggestions, rants (coherent and otherwise), and so on.

We’re always surprised by how many participants take the time to type a response to this kind of question. It may be that site visitors are more likely to do an online study (such as a tree test) when they already have something they want to tell us.

In any case, it’s important that their feedback is collected, cleaned up, and forwarded to the right people in the organization. Typically this involves:

1. Copying the feedback from the online tool to an editable medium (such as a spreadsheet)

2. Deleting obvious garbage responses such as “nothing I can think of” and phrases that don’t have any clear meaning

3. Deleting anything that identifies a participant (unless they explicitly ask to be contacted about their feedback)

4. Grouping or tagging the responses according to an appropriate scheme (e.g. positive vs. negative comments, buying vs. billing vs. support, etc.)

5. Sending the various items to the right people in the organization (e.g. billing responses to the Billing department, support questions to Support, etc.), and asking them to reply directly to the participant where requested.

Next: Chapter 13 - key points