Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • We are ignoring customers who don’t use our website. (For many businesses, such as banks, this may be a big chunk of customers.)

  • We are more likely to get people who visit our site frequently (say, several times a week), but less likely to get people who use our site once a month (e.g. to check their bill).


Which recruitment methods cause it

What causes selection bias?

Here are some common causes of selection bias:

  • Web ads only get web users.
    If we only use web ads to recruit users, then (by definition) we’re only getting those users who visit our website. While this is OK for many studies, it does ignore those customers who use other channels instead of the website. If we need offline users too, we’ll need to find another way to recruit them.

  • Customer email lists only get existing customers.
    If we only use a customer list to email invitations, we are missing prospective customers (a potentially valuable audience) and ex-customers (who are often good sources of honest feedback).
    ~more sources of bias?
  • Daytime in-person studies often get lots of retired people and students.
    These two groups are easier to schedule during typical business hours, but may skew your user sample.
  • Commericial customer panels may be skewed to certain demographics.
    We've used commercial panels that were heavily skewed to consumers rather than businesses, or had a larger-than-normal representation of lower socio-economic groups.
    This is usually easy to solve by being careful in our selection criteria, because panels typically offer lots of ways to select the participants you need.


How can we reduce bias?

We may decide that a given selection bias is acceptable; in our example above, we may only want customers who visit our website, so this implicit selection actually serves as a useful screening mechanism.

...