Summing up the basics



In most cases, we work with others – members of the project team, project owners and sponsors, clients, and vendors. While it’s important that we (as designers and information architects) clearly understand our tree-test results, it’s equally important that we clearly communicate these results to our colleagues and anyone else with a stake in the project.

If we’re working in a fast-paced environment (e.g. in an Agile team or a small organization), the simplest and fastest way to communicate results is an email summary of what we learned and what we do next. In our experience, most stakeholders will not read a long-form report attachment (no matter how lovingly we assembled it), but they will read (or at least skim) a short email of bullet items. Whether we end up writing a full report or not, sending an email summary first is always a good idea.

The summary should include the bare minimum needed to communicate the most important results of the study. Normally this is limited to:

  • A sentence or two describing the study (mainly for those who were not close to it).

  • Bullet lists (or a table) of concisely worded findings and actions

  • A sentence or two outlining what happens next

  • Links to fuller results (results from our online testing tools, more detailed findings, tree spreadsheets, etc.)

  • An invitation for people with more questions/comments/suggestions to contact us.

Note that this summary should be in the body of the email, readable without having to open an attachment or click a link. We must make it as easy as possible for email skimmers (that means most of our audience) to get the gist of the results without any extra effort.


Summary for a single tree

If we tested a single site tree, we may find it useful to group our findings by:

  • Items/ideas that worked well

  • Items/ideas that didn’t work well

  • What will happen next

Subject:  tree test of existing site - summary of results


Hi team,

This week, we ran an online tree test of our existing website, to see if site visitors could find typical items by browsing the headings and subheadings. 87 customers and 92 non-customers participated.

What worked well

  • My Account and About Us were clear, with all participants easily finding relevant items there and not visiting those sections otherwise.
  • The Downloads page in the Support section was easily found by participants looking for our latest software patches.

What needs work

  • The separate Products and Solutions sections were continually confused by both user groups.
  • Existing customers found the SuperWidget product page easily, but non-customers confused it with our Widget Packs and went to those pages instead.
  • The What's New page (where we list our upcoming events) was not found by most participants looking for the next webinar.
  • The FAQ page attracted heavy (and unwanted) traffic even for basic product information.

Next steps

  • Susan will run an online card sort next week to help decide how to reorganize the Products and Solutions content.
  • Lloyd will investigate the FAQ content and see if we can merge it into the main content areas of the site.

Full data and analysis

  • The tree test data and more detailed results are at http://acme.com/studies/tree-test-2016-01-15


As always, contact me if you have any questions, comments, suggestions, or ideas for future studies!


Summary for several trees 
 

If we tested several site trees, we may find it better to group our findings by tree, to keep from confusing our readers:

 

Subject:  tree tests - round 1 - summary of results


Hi team,

This week, we ran online tree tests of two new proposed structures for our website, to see if site visitors could find typical items by browsing the headings and subheadings.

We got a pretty good turn-out of existing customers and non-customers, and a clear winner to use as the skeleton of our site redesign:


Tree 0 (existing site)Tree 1 (task-based)Tree 2 (audience-based)

# of Customers

87

9181
# of non-customers927565
Success rate44%61%74%


What worked well

Tree 1 (task-based)Tree 2 (audience-based)

My Account and About Us were clear, with all participants easily finding relevant items there and not visiting those sections otherwise.

My Account was clear.

The Downloads page in the Support section was found by most participants looking for our latest software patches.The Downloads page (moved to the top level) was found by ALL participants.


What needs work

Tree 1 (task-based)Tree 2 (audience-based)

The separate Products and Solutions sections were still confused by both user groups, as they were in the original site.

The unified Solutions page performed well, attracting the right clicks from both user groups.

Non-customers confused the SuperWidget product page with our Widget Packs and went to those pages instead.

Moving the Pack pages under their corresponding products increased the success rate of the product tasks.


Next steps

  • Susan will run a follow-up tree test combining a revision of Tree 2 with the best elements of Tree 1.
  • Lloyd will start updating content for a Tree-2-based site.


Full data and analysis

  • The tree test data and more detailed results are at http://acme.com/studies/tree-test-2016-01-15


Questions, comments, suggestions, or ideas for future studies? Please contact me.

 



Next: Reporting in more depth



Copyright © 2016 Dave O'Brien

This guide is covered by a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.