If you’re we’re tree-testing on paper, the size of your our tree obviously matters because of the physical effort involved. Most designers would balk at the idea of writing index cards for a tree 7 levels deep with 1000 items or more. They would immediately look for ways to cut their tree down to something more manageable for testing purposes.

You We might think that online testing gets rid of this problem. After all, web apps can now handle vast amounts of data without bogging down, so why not test the whole tree?

The main problem with tree-testing very large structures is not technical; it’s about the participant’s experience of the test. If they have to search through a huge tree (either wide, or deep, or both), it’s going to take them longer for each task, and they’re more likely to get tired of the test sooner. That means more abandoned sessions (“Ack, I’m only halfway through this”) and less chance that they’ll participate in future online studies for you us (“I finished, finally, but I’m not doing one of those again”).

You We can counter this by asking each participant fewer questions, but that requires you us to have more participants in total, which can be a real problem if you’re we’re targeting very specific types of users.

...

The other reason to consider pruning large trees is the effort needed to prepare the tree. If you we can cut a few hundred entries out of your our tree, those are entries that don’t have to be reviewed and cleaned up. If you’re we’re testing several trees, the saved time really starts to add adds up.

If you’re we’re testing a small or medium-sized tree (typically less than 500 items), you can we will normally test the whole tree; it’s not big enough to cause any unwanted test effects. You We still need to check your the tree for specific headings to include/exclude/massage (see the rest of this chapter), but you we can safely skim or skip the rest of this particular section.

If your the tree is large (in our experience, that means 500-1000 items), you we have two options:

• Test the whole tree – This is usually easiest from your our point of view (no pruning required), but can degrade the participant’s experience as described earlier.
  
  
• Test a “pruned” version of the tree – You We may be most interested in how well the top few levels of your our site work, in which case you we can limit your the test tree to the top 3 or 4 levels. Participants should then have time to do a larger number of tasks in a shorter total time.
  You
  We can also cut your the tree down to size by selectively pruning branches that are not important to your our test.
  
  Lastly, you we can opt to test only a particular subtree of the whole tree – e.g., the Sports section of an newspaper site. Here again, there are trade-offs, so it depends on what you’re we’re really looking for.
  
  Each of these methods is explained below.
  
  

Finally, if your our tree is very large (more than 1000 items), it’s almost guaranteed that you’ll we’ll need to prune your tree it to keep your our participants’ efforts from becoming onerous.

...

The simplest way to cut a large tree down to a more manageable size is to cut everything below a certain level. That is, if you we have a tree 5 or 6 levels deep, cut it off at 3 or 4 levels instead (in this example, the topics above the red area):

This is easy, but we must be careful – you’re clearly we’re giving up something up by axing the lower levels. In the full tree, you we would have been able to see if participants could follow the full path down to the exact answer you we intended (and if not, where they strayed off the path). With a “short” tree, you’ll we’ll only find out if they were able to get to the right general area.

This sounds like a big trade-off, but it really depends on the goals of your our particular test. If you’re we’re mainly concerned with the high-level organization of your the site, losing the paths through the lower levels may not matter so much to you, and this type of horizontal pruning may be the best return for your our effort.

If, however, you we do want to know how well your our lower levels perform too, then you we should consider the other methods described below.

...

Instead of pruning straight across a given level of the tree, you we may want to be more selective about what you we cut. There’s no hard-and-fast rule that says you we need to have an equal number of levels across all sections of your our site. If there’s a section that you we don’t intend to test (either because it gets very little traffic, or you we already know that it works well), you we can chop it off short, while leaving other sections (the ones you we really want to test) at their full depth of topics.

For example, suppose you’re we’re testing a new tree for a large corporate site, and you’ve decide we’ve decided to focus your our testing on the sections that handle selling products and supporting them. Those sections might go 4, 5, or even 6 levels deep, which will give you us some detailed results.

Suppose you we also have an About Us section that’s equally deep. If you’re we’re not really interested in testing it (perhaps because that section gets infrequent traffic), you may want to we could cut it off at, say, level 2 (at topics like Company History, Staff, Our Vision, Jobs, etc.).

You We may also want to do some selective pruning after you’ve we’ve written your our tasks (covered in Chapter 7). If there’s a section of your our site that none of your our tasks come anywhere near, you we can probably chop its lower sections without affecting your the results. While it won’t save users time during the test (you we don’t expect them to visit that section anyway), it will save you us time preparing the tree.

If you we have a very large tree (perhaps thousands of items), a more drastic way to cut it down is to only test a single section of it.

Suppose again that you’re we’re testing a large corporate site like Sony or IBM, with many different business divisions, or perhaps a broad government agency with many different departments:

Do you we test the entire tree, or just part of it?

Let’s take the easy case first. If you we want to test several parts of the big tree, you’re we’re going to create tasks that use those sections, so you’ll we’ll clearly need to keep all the sections, at least down to a certain level. If you we want to cut down the tree, you’ll we’ll need to prune levels across the board or selectively, as described above.

If you’re we’re really just testing a single section of the big tree, it may still be a good idea to include all the top-level sections anyway, to see if participants can find their content amid the “clutter” of other divisions/departments. In this case, you we can safely chop off the sections you’re we’re not interested in at level 2 or 3, because you we don’t expect much traffic there (hopefully none at all). Note: don’t cut those sections off at level 1, because participants will quickly figure out that those are just stubs and not where the real action is supposed to be.

For example, if you we want to test Robotics, you we could abbreviate the other divisions:

In the end, you we should only test a subtree by itself if:

• You We don’t intend to test content in other parts of the big tree, and
  
  
• You’re We’re very sure that participants would choose your our subtree every time for each of the tasks you we give them, or that they would go to your our subtree directly in the real site, bypassing the parent site entirely.
  
  

...

Here is their top-level navigation:

If you we were testing pure cycling tasks (look for bicycle bike shoes, check a warranty, read a product FAQ), you we could just test the Cycling subtree (with Shimano Cycling as the root of the tree), because you’d we’d be 99% sure that no one would choose the other sections shown here – they clearly have nothing to do with cycling.

If, however, one of your our tasks involved finding the story of how Shimano got started in the bike industry, now it’s not so cut and dried. You We could imagine some participants going to the Corporate section to find the company’s history. In that case, you we might be best to include these top-level sections, include the full Cycling section (obviously), most or all of the Corporate section, and the top level or two of the Fishing and Rowing sections.

If you we were writing tasks that involved both cycling and fishing, then clearly you we would have include all these top-level sections, but you we could safely cut the Rowing section short. The bigger problem in this case would be that you’re we’re asking each participant to try both cycling and fishing tasks, and a cyclist (for example) may not know anything about fishing, so their fishing results would be suspect. For more on setting different tasks for different user groups, see Different tasks for different user groups in Chapter 7.

...