...
If you run a separate tree test for each user group (as we discussed in ~ on page ~ Different tasks for different user groups in Chapter 7), then you’re going to have a separate set of results for each test, so that’s already taken care of.
...
Suppose, for example, that we were looking at the results for a single tree test of the Shimano website. In Chapter ~ Which part of the tree? in Chapter 6, we saw that Shimano has 3 major types of users – cyclists, anglers, and rowers. And suppose that the cyclists and anglers had no problem with tasks that covered the Corporate section of the site, but rowers failed miserably at those. If you looked at the results of all users together, the high scores of the cyclists and anglers would get mixed in with the low scores of the rowers, and you would just see a mediocre composite score, with no easy way to find out what caused it.
If, however, you could separate the groups and do the same analysis for each group, you could see that it was the rowers who had problems – the other two groups were just fine.
In Adding survey questions in Chapter ~8, “Recruiting participants”, we suggested using survey questions to identify user groups, so we could easily pull them apart later for analysis. Now that we’re ready to analyze the data, we need to:
...
- Run separate tree tests for each cohort.
For example, if we’re analyzing by region, we could run separate tests for region 1, region 2, and region 3.
If we intend to use different tasks for the different groups, then we should definitely run separate tests – see “Different Different tasks for different user groups” groups in Chapter ~7. - Run a single test, using a survey question to identify each factor.
If we’re intending to use the same set of tasks for each cohort, we could add a “Which region do you live in?” survey question, then filter the results using the various values of this question (as described above for user groups).
...