Tree testing, especially the online version of it, is a fairly recent technique, much newer than card sorting, for example. Before tree testing, designers used other methods to check the effectiveness of their site structures.

Here’s our take on how does tree testing compares to a few of these methods.

Closed card sorting

Before tree testing became established, closed card sorts were a popular way to evaluate a site structure.

In a closed sort, participants are given a “pile” of topic cards and asked to put them into pre-named groups. The group names are typically the top-level headings in the site tree:

Open

Once enough participants have done the closed sort, we can inspect the results to see if they put the right cards into the right buckets and (if not) where they disagreed.

While this does help us judge the effectiveness of the top-level headings, the results we get from a closed card sort are not as useful as a tree test, because:

• Most closed sorts only test a single set of groups, which only represent a single level of the tree (usually the top level). Tree testing evaluates the full depth of the tree.

• Closed sorts are essentially a filing exercise (“Where would you put this topic?”). While this is similar to browsing a website, it’s doesn’t mimic the act of browsing as well as tree testing does.

• Tree-test results are easier to analyze than closed-card-sort results (as we’ll see in Chapter 12 - Analyzing results).

For more on comparing closed sorts to tree tests, see David Juhlin's well-considered article, Is closed card sorting an outdated technique for IA?

Usability testing

Usability testing, whether in person or using remote tools, is a powerful technique, but it differs in some major ways from tree testing:

• Usability testing requires a website (or some kind of prototype – paper or clickable). Tree testing can be done earlier in the process because all it really requires is a text skeleton of a site tree.

• Tree testing isolates the organization of topics and their labelling. Usability testing, on the other hand, shows the effects of combining these factors with others such as navigation, search, actual content, visual design, and so on.
  Early in the design process, it’s good to simplify things by just working on a few factors at a time and getting them right. Tree testing lets us do that.

• Tree-testing tools largely automate the analysis of the results. Usability testing usually yields rich (but “analog”) results that take more time and effort to interpret.

Tree testing certainly does not take the place of usability testing. Think of tree testing rather as a complement to usability testing - a early way of testing structural ideas before we even have a prototype.

It's also good to know that that tree-testing results line up nicely with later usability testing, as described in this 2014 academic paper.

Web analytics

Analytics are great because they tell us what our total population of site visitors is actually doing on our site – where they click, where they don’t, and so on.

The traditional weakness of analytics is that, while they can tell us (in detail) what actions our users are taking on the site, they can’t tell us why. When we track visitors jumping from our home page to an intermediate landing page to a content page, we have no idea why they’re going there; maybe they’re looking for what we intended, maybe they’re not. Maybe they leave because they found the answer they wanted; maybe they leave because they didn’t. We have no context, so we don’t know how well the site performed in that sense.

Like a usability test, tree testing gives us context by letting us set specific tasks for the user to do. When they go down a certain path in our tree, we know if that’s a success or a failure.

Analytics are a great way to see where our users are going, and can reveal issues in our site structure, but on their own they’re not enough.

Next:  Chapter 3 - key points