...

•  split this this into separate pages?

...

Primetime for tree testing is early in the design phase, once we’ve done enough research to feel we have a good handle on our audiences, their background, and their needs.

...

By “going wide”, we raise our chances of hitting on the best design. We start with several candidate trees, then use a reliable method (tree testing) to choose the best one.

Consider also that the “other members” that you get input from might include the CEO. And (trust us), if you need to shoot down their out-of-left-field idea, it’s much easier to do that with objective data from testing than just your personal opinion.  J

Perhaps most importantly, testing lots of ideas early on avoids the peril of genius design. By that, we mean designers who believe they are talented, and tend to believe that all their ideas are good ones.

If you have a “genius” designer on the project, tread very carefully. Because when a designer falls in love with a single idea early on, it’s really hard for them to get away from it.

Fixing on a single idea too early is not a new problem. Other industries encountered this (and solved it) years ago.

...

• Brainstorm dozens of ideas
  
  
• Explore 5-10 in more detail
  
  
• Pitch the top 3 to the client
  
  

It brings to mind the advertising person who said that if they went away and worked and then came back with a single idea to pitch, they would be fired on the spot.

...

•  WUD slides - Meridian

...

Making several site trees compete against each other is great, but at some point, we need to reduce them down to a single high-performing tree.

Once we’ve run our first round of tree tests, on 2 or 3 of the most promising trees we thought up, we analyse the results. Typically we find:

• One of the trees performed the best, but it still has some problems to solve.
  
  
• Some of the trees just didn’t work for our participants. We can safely discard these trees and move on.
  
  
• Some of the lower-performing trees may have elements (certain groupings or terms) that actually did perform well, so it may make sense to incorporate these elements into our best tree.
  
  
• (In rare cases, we may find that none of the initial trees perform well enough to continue with. We’ll have to come up with some new ideas quickly (perhaps ideas that were discarded earlier) or go back and do more basic research with users.)
  
  

After the first round of tests, we either have:

• 2 trees that performed reasonably well, but should perform better with the revisions we have in mind, or
  
  
• 1 tree that clearly out-performed the others, and should improve further with our revisions.
  
  

We can now run a second round of tests to see if our revisions did indeed make things better:

• If we tested 2 trees, we can then determine which performed better and which we’re more comfortable going with as our actual site tree.
  
  
• If we tested 1 tree, we can look for any remaining areas or terms that need tweaking.
  
  

In a perfect world, we would keep testing until we had a perfect tree, but there is never time or budget enough for that. We typically only do more than 2 rounds of testing if there are important parts of the tree that are still not performing well enough. 

If you were expecting that tree testing to be a one-off trick – build a tree, test it, and you’re done – you may be alarmed that we recommend several rounds of testing, with several trees, winnowing and refining them until we get a single high-performing tree.

What makes this approach feasible is that we now have mature testing tools that are both:

• Cheap (affordable for any development team), and
  
  
• Fast (able to deliver results in a week or two)
  
  

The combination of cheap and fast changes how we should approach design. Instead of doing a single round of deluxe in-person testing (or worse, no testing at all), we can do several cheap online tests in less time and for less money.

There are cases where in-person testing is the way to go, particularly for complex interactions, or for when there are only a small number of participants available. But for most projects, several rounds of lightweight tests are a better bang for the buck.

And that means we can go wide at the start, and go deep through to the end.

...

Next: Putting it all together

...