Use of the Checklist Method for Content Evaluation of Full-Text Databases: An Investigation of Two Databases Based on Citations from Two Journals
Nisonger, Thomas E., Library Resources & Technical Services
Following a detailed (but not comprehensive) review of the use of citation data as checklists for library collection evaluation, the use of this technique for evaluating database content is explained. This paper reports an investigation of the full-text and indexing and abstracting coverage of Library Literature & Information Science Full Text and EBSCOhost Academic Search Premier, based on checking citations to journal articles in the 2004 volumes of Library Resources & Technical Services and Collection Building. Analysis of these citations shows they were predominately to English-language library and information science journals published in the United States, with the majority dating from 2000 to 2004. Library Literature & Information Science Full Text contained 21.1 percent of the citations in full-text format, while the corresponding figure for Academic Search Premier was 16.1 percent. The database coverage also is analyzed by publication date, country of origin, and Library of Congress classification number of cited items. Some limitations to the study are acknowledged, while issues for future research are outlined.
That the librarianship paradigm is rapidly changing with the evolution from a print to an electronic environment is almost a cliche. Relatively new formats, such as full-text databases, electronic journals, electronic books, and the Web, offer numerous challenges to contemporary librarians, including a need for evaluation techniques. While a host of generally accepted collection evaluation methods were developed for the twentieth century's relatively stable, mostly print environment, identifying appropriate evaluation methodologies ranks among the library profession's major challenges in the first decade of the twenty-first century. As will be illustrated in the following literature review, the checklist method, dating to the mid-nineteenth century, is one of the oldest and among the most often used approaches to library collection evaluation. This paper's purpose is to demonstrate the use of a citation-based checklist approach by evaluating the content of two full-text databases: Library Literature & Information Science Full Text and Academic Search Premier.
The Guide to the Evaluation of Library Collections offers a succinct definition of the checklist approach: "With this procedure the evaluator selects lists of titles or works appropriate to the subjects collected, to the programs or goals of the library, or to the programs and goals of consortia. These lists are then searched in the library files to determine the percentage the library has in its own collection." (1) More specifically, the lists are checked in the library's catalog (originally a card catalog, now an online public access catalog [OPAC]).
The benefits and drawbacks associated with the checklist technique have been discussed in the literature by Lockett, Lundin, and the author, among others. (2) On the positive side, lists can be compiled to meet the needs of a particular library or type of library and they can be examined to increase knowledge of the literature. Lists also are straightforward to implement, require little subject expertise, and provide objective data that is easily understood. On the negative side, the collection might hold other resources better than those on the list; all items on the list are not of equal value; appropriate lists might be difficult to locate; held items might not be available because they are checked out, missing, or for other reasons; and many lists focusing on a single subject area do not consider resources from other disciplines. One of the more compelling criticisms is the fact that the checklist approach was developed to test ownership in the traditional model of librarianship and usually does not consider items obtained on interlibrary loan or licensed electronically.
History of the Checklist Method
According to Mosher and other authorities, the earliest reported collection evaluation in an American library, published in 1849, used the checklist method. …