Collecting task ideas

As we come up with ideas for tasks, it’s important to track them against our tree. The simplest way is to jot them down in our spreadsheet, in the Task column:

Open

In this first pass, the goal is to collect our ideas quickly.

• Don’t sweat the wording yet. At this stage, we’re just compiling a list of task ideas, some of which we won’t end up using. Trying to fine-tune wording now may be wasted effort, and it will slow us down early in the process.

• Don’t worry about coverage yet. If we come up with 5 task ideas for a single small area of our site, that’s OK. It means we’ll have 5 candidates to choose from later when we’re trimming the task list. And if we don’t have any ideas for a given section, that’s OK too – we’ll get a chance to fill that in when we check task coverage later.

• Get everyone involved. If we’re working with a team, we can distribute the work. That makes everything go faster, and gets everyone engaged with the testing. This is particularly easy if we use a shared online spreadsheet.

Refining our task list

After we’ve taken a first pass through the tree, either by ourselves or with others contributing too, we’ll likely come up with too many task ideas. This is a good thing, because we can then refine our task list down to a smaller number of focused items.  

To pick the winners for our task list, we look for:

• Tasks that test our stated goals. When we first planned this test, we wrote down the top things we wanted to find out. How well does each task idea answer those questions?

• Tasks that are the most common, critical, and suspect. See Which tasks to include? earlier in this chapter.

• Tasks that provide good coverage for the parts of the tree that we want to test. See Checking coverage below.

It's important for us to mark our task ideas to show which we're keeping and which we're not. For example, we could highlight the chosen ones in green, or we could leave them unmarked and highlight the not-chosen ones in red. It doesn’t matter which method we use, as long as we’re consistent (and we include a legend so that others can understand our method later.)

Using the spreadsheet method, we find it’s better to mark tasks as deleted (e.g. using strike-through or a red highlight) rather than delete them outright. That way, if we change our minds later, we can reinstate a task that we initially “crossed out”. In the example below, we’ve highlighted the chosen tasks in green and struck out the unchosen tasks:

Open

Checking coverage

When deciding which tasks to include, we also want to keep an eye on coverage - which parts of the tree we’re testing, and which we’re ignoring.

It’s not usually feasible to test every subtree of every section of the site, and usually it’s not necessary either. For example, if participants can find the specifications for product A in our site (by looking in the Products section, say), it’s safe to assume that they can also find product B’s specs in the same section.

Once we’ve created tasks that cover the most common, critical, and contentious areas of the site, it’s a good idea to pull back and see which parts of the tree are covered too much or too little.

• If we are using the spreadsheet method to refine our tree and write the tasks, checking task coverage is simply a matter of scrolling down the tree and looking for entries in the Task column. We can see which sections of the tree are “covered” by tasks, and which aren’t.

• If our tree-testing software checks coverage for us, we can activate that feature and see where the gaps are.

If we find gaps in coverage (and we likely will), we may want to change up our tasks a bit:

• If a given section is particularly important to test, we will want at least one task for it, and maybe even two. If we add too many tasks to a given section, however, participants may “learn” that part of the tree better than they would during a real visit to the site, and skew our results.

• If a section is not important to test (e.g. it worked well in the past, and we’re not changing it now), it may be OK to have no tasks for it. Note, however, that if we change the tree around it, this might affect how users then navigate this area of the site. If we have any doubts about this, we should consider adding a task to make sure.

Keep in mind that some of our tasks may have several correct answers in different parts of the site (or at least several places where we suspect participants will look). If we’re using a spreadsheet to track tasks, it’s often a good idea to copy our task ideas into these alternate slots too. We typically use some kind of styling (e.g. italics) to indicate that these are references to a task mentioned elsewhere in the tree.

Next: Different tasks for different user groups