...
We should also reconsider tasks that (almost) everyone got right. It’s great that participants easily found what they were looking for (after all, that’s our Big Goal), but first let’s make sure those tasks weren’t “gimme’s”. Did we give away the answer by careless word matching, or by phrasing the question in the same browsing sequence that the participant would follow in the tree? (See Chapter ~ for For more on these problems, see Writing a good task in Chapter 7.)
Even if the task passes these tests, we still may want to replace it because (unless we change the parts of the tree that it tests) we won’t learn much from it in later testing. It works, and we should move on. This is especially true if we find that another part of the tree is either not performing well or is not being tested enough; it may be better to replace our “golden” task with one that tells us more about what needs improving. ~but replacing tasks will make before/after comparisons harder to make, so is this really a good idea? Just add the more useful tasks and keep the golden ones?
...