Table of Contents | ||
---|---|---|
|
...
Size does matter
If we’re tree-testing on paper, the size of our tree obviously matters because of the physical effort involved. Most designers would balk at the idea of writing index cards for a tree 7 levels deep with 1000 items or more. They would immediately look for ways to cut their tree down to something more manageable for testing purposes.
...
The other reason to consider pruning large trees is the effort needed to prepare the tree. If we can cut a few hundred entries out of our tree, those are entries that don’t have to be reviewed and cleaned up. If we’re testing several trees, the saved time really adds up.
How big is the tree?
If we’re testing a small or medium-sized tree (typically less than 500 items), we will normally test the whole tree; it’s not big enough to cause any unwanted test effects. We still need to check the tree for specific headings to include/exclude/massage (see the rest of this chapter), but we can safely skim or skip the rest of this particular section.
If the tree is large (in our experience, that means 500-1000 items), we have two options:
Test the whole tree – This is usually easiest from our point of view (no pruning required), but can degrade the participant’s experience as described earlier.
Test a “pruned” version of the tree – We may be most interested in how well the top few levels of our site work, in which case we can limit the test tree to the top 3 or 4 levels. Participants should then have time to do a larger number of tasks in a shorter total time.
We can also cut the tree down to size by selectively pruning branches that are not important to our test.
Lastly, we can opt to test only a particular subtree of the whole tree – e.g., the Sports section of an newspaper site. Here again, there are trade-offs, so it depends on what we’re really looking for.
Each of these methods is explained below.
Finally, if our tree is very large (more than 1000 items), it’s almost guaranteed that we’ll need to prune it to keep our participants’ efforts from becoming onerous.
Pruning levels
The simplest way to cut a large tree down to a more manageable size is to cut everything below a certain level. That is, if we have a tree 5 or 6 levels deep, cut it off at 3 or 4 levels instead (in this example, the topics above the red area):
...
This is easy, but we must be careful – we’re giving up something by axing the lower levels. In the full tree, we would have been able to see if participants could follow the full path down to the exact answer we intended (and if not, where they strayed off the path). With a “short” tree, we’ll only find out if they were able to get to the right general area.
...
If, however, we do want to know how well our lower levels perform too, then we should consider the other methods described below.
Pruning unnecessary branches
Instead of pruning straight across a given level of the tree, we may want to be more selective about what we cut. There’s no hard-and-fast rule that says we need to have an equal number of levels across all sections of our site. If there’s a section that we don’t intend to test (either because it gets very little traffic, or we already know that it works well), we can chop it off short, while leaving other sections (the ones we really want to test) at their full depth of topics.
...
Suppose we also have an About Us section that’s equally deep. If we’re not really interested in testing it (perhaps because that section gets infrequent traffic), we could cut it off at, say, level 2 (at topics like Company History, Staff, Our Vision, Jobs, etc.).
...
We may also want to do some selective pruning after we’ve written our tasks (covered in Chapter 7). If there’s a section of our site that none of our tasks come anywhere near, we can probably chop its lower sections without affecting the results. While it won’t save users time during the test (we don’t expect them to visit that section anyway), it will save us time preparing the tree.
Testing subtrees
If we have a very large tree (perhaps thousands of items), a more drastic way to cut it down is to only test a single section of it.
Suppose again that we’re testing a large corporate site like Sony or IBM, with many different business divisions, or perhaps a broad government agency with many different departments:
...
Do we test the entire tree, or just part of it?
...
For example, if we want to test Robotics, we could abbreviate the other divisions:
...
In the end, we should only test a subtree by itself if:
We don’t intend to test content in other parts of the big tree, and
We’re very sure that participants would choose our subtree every time for each of the tasks we give them, or that they would go to our subtree directly in the real site, bypassing the parent site entirely.
A real-life example makes this clearer. Suppose we’re testing the Shimano website. Shimano is best known for cycling products (bike pedals, gears, etc.), but they also make equipment for fishing and rowing.
Here is their top-level navigation:
...
If we were testing pure cycling tasks (look for bike shoes, check a warranty, read a product FAQ), we could just test the Cycling subtree (with Shimano Cycling as the root of the tree), because we’d be 99% sure that no one would choose the other sections shown here – they clearly have nothing to do with cycling.
If, however, one of our tasks involved finding the story of how Shimano got started in the bike industry, now it’s not so cut and dried. We could imagine some participants going to the Corporate section to find the company’s history. In that case, we might be best to include these top-level sections, include the full Cycling section (obviously), most or all of the Corporate section, and the top level or two of the Fishing and Rowing sections.
...
If we were writing tasks that involved both cycling and fishing, then clearly we would have include all these top-level sections, but we could safely cut the Rowing section short. The bigger problem in this case would be that we’re asking each participant to try both cycling and fishing tasks, and a cyclist (for example) may not know anything about fishing, so their fishing results would be suspect. For more on setting different tasks for different user groups, see Different tasks for different user groups in Chapter 7.
...