Academic journal article Journal of the Medical Library Association

Impact Factor: A Valid Measure of Journal Quality?

Academic journal article Journal of the Medical Library Association

Impact Factor: A Valid Measure of Journal Quality?

Article excerpt

Objectives: Impact factor, an index based on the frequency with which a journal's articles are cited in scientific publications, is a putative marker of journal quality. However, empiric studies on impact factor's validity as an indicator of quality are lacking. The authors assessed the validity of impact factor as a measure of quality for general medical journals by testing its association with journal quality as rated by clinical practitioners and researchers.

Methods: We surveyed physicians specializing in internal medicine in the United States, randomly sampled from the American Medical Association's Physician Masterfile (practitioner group, n = 113) and from a list of graduates from a national postdoctoral training program in clinical and health services research (research group, n = 151). Respondents rated the quality of nine general medical journals, and we assessed the correlation between these ratings and the journals' impact factors.

Results: The correlation between impact factor and physicians' ratings of journal quality was strong (r^sup 2^ = 0.82, P = 0.001). The correlation was higher for the research group (r^sup 2^ = 0.83, P = 0.001) than for the practitioner group (r^sup 2^ = 0.62, P = 0.01).

Conclusions: Impact factor may be a reasonable indicator of quality for general medical journals.

INTRODUCTION

The impact factor of a journal reflects the frequency with which the journal's articles are cited in the scientific literature. It is derived by dividing the number of citations in year 3 to any items published in the journal in years 1 and 2 by the number of substantive articles published in that journal in years 1 and 2 [I]. For instance, the year 2002 impact factor for Journal X is calculated by dividing the total number of citations during the year 2002 to items appearing in Journal X during 2000 and 2001 by the number of articles published in Journal X in 2000 and 2001 (Figure 1). Conceptually developed in the 1960s, impact factor has gained acceptance as a quantitative measure of journal quality [2]. Impact factor is used by librarians in selecting journals for library collections, and, in some countries, it is used to evaluate individual scientists and institutions for the purposes of academic promotion and funding allocation [3, 4]. Not surprisingly, many have criticized the methods used to calculate impact factor [5, 6]. However, empiric evaluations of whether or not impact factor accurately measures journal quality have been scarce [7].

The use of impact factor as an index of journal quality relies on the theory that citation frequency accurately measures a journal's importance to its end users. This theory is plausible for journals whose audiences are primarily researchers, most of whom write manuscripts for publication. By citing articles from a given journal in their own manuscripts, researchers are in essence casting votes for that journal. Impact factor serves as a tally of those votes.

A journal's impact within clinical medicine, however, depends largely on its importance to practitioners, most of whom never write manuscripts for publication and thus never have a chance to "vote." Citation frequency may therefore better reflect the importance of clinical journals to researchers than practitioners. Because the opinions of both practitioners and researchers are relevant in judging the importance of clinical journals, the validity of impact factor as a measure of journal quality in clinical medicine is uncertain. The authors therefore sought to examine whether impact factor is a valid measure of journal quality as rated by clinical practitioners and researchers.

METHODS

As part of a study assessing the effect of journal prestige on physicians' assessments of study quality [8], we mailed questionnaires to 416 physicians specializing in internal medicine in the United States. We recruited half of our sample (practitioner group) from the American Medical Association's (AMA's) master list of licensed physicians. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.