Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

What is selection bias?

From Wikipedia’s article:

Selection bias is the selection of individuals, groups or data for analysis in such a way that proper randomization is not achieved, thereby ensuring that the sample obtained is not representative of the population intended to be analyzed.

 

Basically, it means that In other words, certain recruitment methods may yield a skewed selection of participants, rather than the representative sample that you wantednormally want.

For example, suppose that you only use a web ad on your site to recruit for your study. Only people who visit your site in the next few days (the duration of your recruitment) will see the ad. This means that:

...

  • Web ads only get web users
    If you only use web ads to recruit users, then (by definition) you’re only getting those users who visit your website. While this is OK for many studies, it does ignore those customers who use other channels instead of the website. If you need offline users too, you’ll need to find another way to recruit them.

  • Customer email lists only get existing customers
    If you only use a customer list to email invitations, you are missing prospective customers (a potentially valuable audience) and ex-customers (who are often good sources of honest feedback).

  • ~more sources of bias?


How can we reduce bias?

You may decide that a given selection bias is acceptable; in our example above, you may only want customers who visit your website, so this implicit selection actually serves as a useful screening mechanism.

...