The time, effort, money, and participants it will take to develop your site tree depends partly on how many rounds of testing you’re intending to do. More rounds usually means a better result (as you would expect), but there are also diminishing returns to consider.
For most projects we work on, 2 rounds of tree testing is standard:
- Round 1 - test the existing tree and revised tree(s)
- Round 2 – test a “best of” hybrid of the earlier trees
The first round of testing shows you where your tree is doing well (yay!) and where it needs more work. So you make some thoughtful revisions. Careful, though, because even if the problems you found seem to have obvious solutions, you still need to make sure your revisions actually work for users, and don’t cause further problems.
The good news is, it’s dead easy to run a second test, because it’s just a small revision of the first one. You already have the tasks and all the other bits worked out, so it’s just a matter of making a copy of the test (in whatever tool you’re using), pasting in your revised tree, and hooking up the correct answers. In an hour or two, you’re ready to pilot it again (to err is human, remember) and then send it off to a fresh batch of participants.
There are two possible outcomes here:
- Your fixes are spot-on, the participants find the correct answers more frequently and easily, and your overall score climbs. You could have skipped this second test, but confirming that your changes worked is both good practice and a good feeling. It’s also something concrete to show your boss.
- Some of your fixes didn’t work, or (given the tangled nature of IA work) they worked for the problems you saw in round 1, but now they’ve caused more problems of their own. Bad news, for sure, but better that you uncover them now in the design phase (when it takes a few days to revise and re-test) instead of further down the track when the IA has been signed off and changes become painful.
Note that Round 1 combines the “before” and “after” testing, because most of our clients have a good idea of where the weaknesses are in their existing tree. If you don’t, we recommend a “Round 0” where you test the existing tree; this could be combined with an open card sort to help generate ideas for the revised structure.
On some larger and more complex trees, additional revision rounds may be needed to confirm we have solved the major issues we uncover.
For planning, this means that you need to:
- add the desired # of rounds into your project schedule
- determine how you will get enough fresh participants for each round
Next: Which trees will you test?