14 - Revising and retesting

“I may not have gone where I intended to go, but I think I have ended up where I needed to be.” ― Douglas Adams

Now that we’ve completed a round of tree testing, we should have lots to act on – revisions to trees that performed well, discarding of trees that just didn’t work, identifying which elements to pursue and which to leave behind, tweaking of terms that participants had trouble with, and so on.

If we found a tree that tested extremely well, and only needs very minor tweaks, we may be “done” as far as tree testing is concerned. More likely, though, we’ll want to make some major decisions and revisions, and those will need retesting to make sure our changes were the right ones.

Retesting also provides two big wins to the organization, beyond a better site tree:

  • It shows that the organization is willing to try out ideas, have some of them fail early (before they reach the production site), and use that feedback to come up with something better. This “room to fail” is a key to real innovation.

  • It lets the project team “prove” that their final design is a good one, not just by hand-waving, but by showing objective before-and-after data.


Revising trees

Single tree vs. competing trees, keeping a record, no pandering

Cherrypicking and hybrid trees

Selecting the best from each tree

Rewording and replacing tasks

Fixing misunderstandings, retiring high-success tasks, updating answers

Tuning survey questions

If we didn't get them quite right the first time

Using fresh participants

Avoid reusing participants unless the pool is small

When are we done?

70% success and no more substantial revisions

 Key points


Next: Chapter 15 - Special considerations


Copyright © 2024 Dave O'Brien

This guide is covered by a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.