...

If you have access to a large list of customers (perhaps thousands), you may be tempted to email them all and get lots of results fast.

...

First of all, you should almost never email everyone in a big list. Lists of those size usually have more detail in them that you can use to filter the list down to the people you really want (not just bank customers, for example, but those with home loans who use Internet banking). For more on this, see “Filtering lists” Filtering lists below.

Second, even if you still have a big list after filtering (lucky you), remember that people have a limited appetite for invitations from a given organization. If you or anyone else in your organization wants to run another study in a month or two (remember that we recommend at least 2 rounds of tree testing to get it right), you should probably avoid emailing the same people you just pinged this week. Many large organizations have formal rules about this, typically along the lines of “Do not email a given customer more than once every 3 months”. Even if your organization has no such policy, it’s still a healthy rule of thumb.

...

• If you need results fast, and you have users to burn (so to speak), invite larger batches of users.
  
  
• If you don’t have many users on your list, and so need to conserve them, invite smaller batches so you can get just enough participants to show clear results.
  
  
  

You don’t just want any 50 people to do your study; you want the right 50 people – people who match your idea of a representative user.

...

Often, however, you may want to get more specific about who does your study. If you are reorganising reorganizing the Large Business section of a bank’s website, for example, you want large-business users to test the new structure, but you don’t want personal-banking users because they do different tasks, use different terminology, and would generally be irrelevant to your study.

...

• Early filtering – You only send invitations to people who fit your criteria (by filtering a customer database first).
  
  
• Late filtering – You invite anyone to do your study (via a blanket email blast or a web ad), then screen out the people you don’t want (by using screening questions just before the tree test starts). For more on this, see “Screening who gets to do the test” below Screening for specific participants later in this chapter.
  
  
  

We mentioned earlier than inviting people by email is intrusive – most people have a limited appetite for unsolicited invitations, and some people may not want to be contacted at all. We need to respect their wishes and keep their goodwill.

...