Web-Based Tools for Citation Data Management: Research Is Big Business and a Key to National Rankings and International Status
Herther, Nancy K., Searcher
Private research corporations and well-endowed private universities vie for funding and the prestige that comes with it. Between 1998 and 2003, for example, the National Institutes of Health (NIH) doubled its science funding. However, about 5 years ago, research funding became stagnant, or even declined, creating an increasingly competitive marketplace for dwindling resources (1). At the same time, due to economic pressures and globalization, research itself has become a primary factor in competitive positioning--for individual researchers, their institutions, and even for national standings in our global community.
Citation analysis has become a critical measure of research value, whether for grant applications, formal promotional processes, or just to enhance public perception of individuals or institutions (2). This has significantly increased the use of Thomson's Web of Science (WoS)--and other citation sources--to gather data on "citedness," using this to rank the output of researchers and the relative importance of their institutions or the journals in which they publish.
Increasing Sources of Citations
The increased desire to use citedness to measure value and relative positioning is spurring the development of new databases and software--from Elsevier's Scopus, CiteSeer, and other citation-based products--to the development of cited references in Google Scholar and, increasingly, the inclusion of cited reference information in scholarly databases from such vendors as CSA and EBSCO (3).
Certainly the increasing number of databases and websites with citation data has become significant. These resources make it possible to go far beyond the 9,000-plus journals indexed in WoS. Although many purists would question whether or not looking comprehensively for all potential citations makes sense, in practice, most clients want whatever they can get.
Using citation information as a gauge of quality research is not without criticism. Even more troubling is the fact that there is no standard, accepted method, or yardstick for analyzing the productivity and importance of researchers, their published works, or their institutions (4).
Is More Always Better?
Anyone who has worked on citation analysis--whether for competitive intelligence, marketing, or academic assessment or for more traditional bibliometric analysis--has learned the inherent difficulties with the process. The difficulties can range from making sure that you have everything on your person (common names are notoriously troublesome) to achieving a confidence level in terms of the universe of potential cited references. Good citation analysis depends on the integrity of the reference lists, which, in turn, can depend on such apparently minor issues as the quirks of the various style manuals used by the different journals.
More data adds significant complexity to the task. If the data comes from a single source, you can at least be assured that the field tags will be common. But what about large data sets from various resources? Merging these to create a single database and working with all the potential formulae you might need to reach references is a major problem for researchers.
Last year, two information science faculty did a citation assessment of their departmental colleagues using the WoS, Elsevier's Scopus, and Google Scholar (GS). The results were sobering: "The WoS data took about 100 hours of collecting and processing time, Scopus consumed 200 hours, and GS a grueling 3,000 hours" (5). This example could be considered as close to ideal as possible: The faculty were in the same field, the same department, and actively cooperated with the research. Imagine the difficulties added when one is doing competitive research without this level of buy-in? Thankfully, there are some good sources of assistance and useful products entering the marketplace. …