...
- 30 participants will start showing patterns in your results, but it will be hard to know what to do with “small effects” because you don’t have enough participants to know if these are outlier results that can be ignored.
- 50-100 participants will make the patterns much clearer, and you’ll be able to identify which results are significant and which can be discarded.
- Hundreds of participants give diminishing returns, and you’re potentially “using up” participants who would be better employed in a subsequent round of testing.
For a more rigorous look at how many participants you should aim for, see this MeasuringU article on tree testing.
Counting by user group
Most products/websites have more than one major type of user. In Chapter ~, for example, we saw that the Shimano website has 3 user groups – cyclists, anglers, and rowers.
...