Vol. 10, No. 1
How to Recruit Usability Test Participants
First, a little digression on sample size and test validity
Usability tests are only valid if the participants (test subjects) are drawn from a representative sample. Sampling allows the researcher (in this case, the usability test administrator) to generalize findings from a subset onto the general population. Most people are familiar with random sampling, but this is not the best method for recruiting pariticipants in a usability test, because visitors to a particular site or users of a particular online product tend to have key things in common—what I call defining characteristics. These are the characteristics that the participants must have in common for the test results to be relevant and so that you can generalize the results.
The sampling method for a usability test needs to balance the cost of adhering to the method against a) the level of precision needed for the study, and b) the time it takes to analyze the study data. While some usability testing specialists might advocate seeking only statistically significant results, I advocate using the most cost efficient, yet sufficient number of participants. Jakob Nielsen and other usability experts found that over 80% of a site or product's functional deficiencies can be found with as few as 5-7 participants. Due to their cost- and time-effectiveness, I advocate using this number of participants for "initial assessment" tests (early in the design process). If time is available, and for the sake of garnering more thorough feedback, a larger number of participants may be used for "ongoing validation" tests (after the Website or product has been developed and deployed).
Finally, on to talking about finding participants
Given that you need only 5-7 participants for your initial test, it is vital that you solicit participants that have all of the "defining characteristics" of the real Website or product users. Examples of three defining characteristics are:
The specific set of defining characteristics will vary from application to application. To determine them with confidence, you will need to speak to people in your organization that are closest to the users, if not the users themselves. (Be careful not to just take your management's opinions of key user characteristics; make sure that these characteristics are substantiated with real customer or prospective customer data). False assumption really can harm you; take care to avoid them.
Regarding previous exposure to the Website or product
In general, test participants should have had no prior exposure to the Website or product being tested (i.e., have never seen or used it). If internal people "know too much" or may potentially give biased feedback, you will need to solicit people outside the company. If you're planning a series of tests for an existing product that does require participants that have previously used the site or product, sales/marketing customer lists are probably your best resource. Sales managers are often willing to contact clients, and ask them if they know users with the defining characteristics who would be willing to participate in your test.
If the product is new, it is usually necessary to use the following methods to find test participants:
This should give you enough guidelines to get started. Remember, you can use fewer participants, but, if you want to get results that you can confidently communicate to your development group, just make sure you pick your participants from people "swimming in the right participant pool." Also, remember that perseverance pays off—you may sometimes have to actively recruit 12-15 people to find 5-7 who are available to participate in your test.
Want to comment on this article? Please write Mark.
Copyright 2002 HallMark Consulting All rights reserved