Testing the Database of International Rehabilitation Research: Using Rehabilitation Researchers to Determine the Usability of a Bibliographic Database
Munger, Heather L., Journal of the Medical Library Association
Objectives: This study tested the usability of the Database of International Rehabilitation Research, a bibliographic database developed by the Center for International Rehabilitation Research Information and Exchange (CIRRIE).
Methods: Potential users, i.e., rehabilitation researchers, were asked to participate in a usability study. Test questions were designed to represent common tasks performed in a bibliographic database. Participants were asked to think aloud during the test so that both their actions and comments could be recorded.
Results: This study identified common problems that participants had while searching the database and aspects of the database that needed improvement.
Conclusions: Usability testing proved to be an effective method for evaluating database effectiveness and user satisfaction. The method used provided valuable information about how the database searchers approached their searches as well as how they performed them.
Usability is a major consideration for end-user databases because the success of most users' searches depends, in part, on how usable the system is. The Oxford English Dictionary defines usable as "that may or can be used; capable of use" . Evaluating the usability of a bibliographic database can be accomplished by employing usability testing methods. For the purpose of this usability test, rehabilitation researchers were observed completing common tasks performed in a bibliographic database.
The Center for International Rehabilitation Research Information and Exchange (CIRRIE) at the University at Buffalo, State University of New York, is supported through a grant from the National Institute on Disability and Rehabilitation Research (NIDRR), Department of Education. The mission of CIRRIE is to facilitate the sharing of information and expertise in rehabilitation research between the United States and other countries. One of CIRRIE's primary objectives is the development and maintenance of a subscription-free bibliographic database of references to published reports of rehabilitation research conducted outside of the United States. To date, comprehensive searching of rehabilitation research has been very difficult for a variety of reasons, including selective coverage of rehabilitation literature by existing databases, the breadth and interdisciplinary nature of the field, and the fact that existing relevant databases are not accessible to all in the research community. The CIRRIE Database of International Rehabilitation Research* complements the National Rehabilitation Information Center's database REHABDATA, which consists of disability and rehabilitation research that has been published within the United States.
By the spring of 2001, the bibliographic database had been developed, the interface designed and implemented, and the decision made to test its usability. At that point, CIRRIE staff contacted the author, who had conducted other usability studies and was unaffiliated with CIRRIE. This paper reports the application of the principles of usability testing to the CIRRIE Database of International Rehabilitation Research.
The literature provides several descriptions of usability testing methods. Nielsen measures usability "by having a number of test users (selected to be as representative as possible of the intended users) use the system to perform a prespecified set of tasks" . Rubin defines usability testing as "a process that employs participants who are representative of the target population to evaluate the degree to which a product meets specific usability criteria" . Dumas and Redish describe usability testing methods as sharing five characteristics:
1. The primary goal is to improve the usability of a product. Planning the test requires that specific goals and concerns be articulated for each test.
2. The participants represent real users.