Summing up the basics


In most cases, we work with others – members of the project team, project owners and sponsors, clients, and vendors. While it’s important that we (as designers and information architects) clearly understand our tree-test results, it’s equally important that we clearly communicate these results to our colleagues and anyone else with a stake in the project.

If we’re working in a fast-paced environment (e.g. in an Agile team or a small organization), the simplest and fastest way to communicate results is an email summary of what we learned and what we do next. In our experience, most stakeholders will not read a long-form report attachment (no matter how lovingly we assembled it), but they will read (or at least skim) a short email of bullet items. Whether we end up writing a full report or not, sending an email summary first is always a good idea.

The summary should include the bare minimum needed to communicate the most important results of the study. Normally this is limited to:

  • A sentence or two describing the study (mainly for those who were not close to it).

  • Bullet lists (or a table) of concisely worded findings and actions

  • A sentence or two outlining what happens next

  • Links to fuller results (results from our online testing tools, more detailed findings, tree spreadsheets, etc.)

  • An invitation for people with more questions/comments/suggestions to contact us.

Note that this summary should be in the body of the email, readable without having to open an attachment or click a link. We must make it as easy as possible for email skimmers (that means most of our audience) to get the gist of the results without any extra effort.

Summary for a single tree

If we tested a single site tree, we may find it useful to group our findings by:

  • Items/ideas that worked well

  • Items/ideas that didn’t work well

  • What will happen next

Here’s a sample email that summarizes the results for a single tree:

 

Subject:  tree test of existing site - summary of results

Hi team,

This week, we ran an online tree test of our existing website, to see if site visitors could find typical items by browsing the headings and subheadings. 87 customers and 92 non-customers participated.

What worked well

  • My Account and About Us were clear, with all participants easily finding relevant items there and not visiting those sections otherwise.

  • The Downloads page in the Support section was easily found by participants looking for our latest software patches.

What needs work

  • The separate Products and Solutions sections were continually confused by both user groups.

  • Existing customers found the SuperWidget product page easily, but non-customers confused it with our Widget Packs and went to those pages instead.

  • The What's New page (where we list our upcoming events) was not found by most participants looking for the next webinar.

  • The FAQ page attracted heavy (and unwanted) traffic even for basic product information.

Next steps

  • Susan will run an online card sort next week to help decide how to reorganize the Products and Solutions content.

  • Lloyd will investigate the FAQ content and see if we can merge it into the main content areas of the site.

Full data and analysis

As always, contact me if you have any questions, comments, suggestions, or ideas for future studies!

Summary for several trees 

If we tested several site trees, we may find it better to group our findings by tree, to keep from confusing our readers.

Here’s a sample email for summarizing the results from several trees:

 

Subject:  tree tests - round 1 - summary of results

Hi team,

This week, we ran online tree tests of two new proposed structures for our website, to see if site visitors could find typical items by browsing the headings and subheadings.

We got a pretty good turn-out of existing customers and non-customers, and a clear winner to use as the skeleton of our site redesign:

 

Tree 0 (existing site)

Tree 1 (task-based)

Tree 2 (audience-based)

 

Tree 0 (existing site)

Tree 1 (task-based)

Tree 2 (audience-based)

# of customers

87

91

81

# of non-customers

92

75

65

Success rate

44%

61%

74%

What worked well

Tree 1 (task-based)

Tree 2 (audience-based)

Tree 1 (task-based)

Tree 2 (audience-based)

My Account and About Us were clear, with all participants easily finding relevant items there and not visiting those sections otherwise.

My Account was clear.

The Downloads page in the Support section was found by most participants looking for our latest software patches.

The Downloads page (moved to the top level) was found by ALL participants.

What needs work

Tree 1 (task-based)

Tree 2 (audience-based)

Tree 1 (task-based)

Tree 2 (audience-based)

The separate Products and Solutions sections were still confused by both user groups as they were in the original site.

The unified Solutions page performed well, attracting the right clicks from both user groups.

Non-customers confused the SuperWidget product page with our Widget Packs and went to those pages instead.

Moving the Pack pages under their corresponding products increased the success rates of the product tasks.

Next steps

  • Susan will run a follow-up tree test combining a revision of Tree 2 with the best elements of Tree 1.

  • Lloyd will start updating content for a Tree-2-based site.

Full data and analysis

Questions, comments, suggestions, or ideas for future studies? Please contact me.


Next: Reporting in more depth


Copyright © 2024 Dave O'Brien

This guide is covered by a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.