Academic journal article Journal of the Medical Library Association

Evaluation of PubMed Filters Used for Evidence-Based Searching: Validation Using Relative Recall

Academic journal article Journal of the Medical Library Association

Evaluation of PubMed Filters Used for Evidence-Based Searching: Validation Using Relative Recall

Article excerpt

Objectives: The research sought to determine the value of PubMed filters and combinations of filters in literature selected for systematic reviews on therapy-related clinical questions.

Methods: References to 35,281 included and 48,514 excluded articles were extracted from 2,629 reviews published prior to January 2008 in the Cochrane Database of Systematic Reviews and sent to PubMed with and without filters. Sensitivity, specificity, and precision were calculated from the percentages of unfiltered and filtered references retrieved for each review and averaged over all reviews.

Results: Sensitivity of the Sensitive Clinical Queries filter was reasonable (92.7%, 92.1-93.3); specificity (16.1%, 15.1-17.1) and precision were low (49.5%, 48.5-50.5). The Specific Clinical Queries and the Single Term Medline Specific filters performed comparably (sensitivity, 78.2%, 77.2-79.2 vs. 78.0%; 77.0-79.0; specificity, 52.0%, 50.8-53.2 vs. 52.3%, 51.1-53.5; precision, 60.4%, 59.4-61.4 vs. 60.6%, 59.6-61.6). Combining the Abridged Index Medicus (AIM) and Single Term Medline Specific (65.2%, 63.8-66.6), Two Terms Medline Optimized (64.2%, 62.8-65.6), or Specific Clinical Queries filters (65.0%, 63.6-66.4) yielded the highest precision.

Conclusions: Sensitive and Specific Clinical Queries filters used to answer questions about therapy will result in a list of clinical trials but cannot be expected to identify only methodologically sound trials. The Specific Clinical Queries filters are not suitable for questions regarding therapy that cannot be answered with randomized controlled trials. Combining AIM with specific PubMed filters yields the highest precision in the Cochrane dataset.

INTRODUCTION

PubMed [1] is one of the major sources of medical information. Although information sources containing integrated data, like online textbooks and guidelines, are more practical for handling daily clinical questions, PubMed or comparable databases are indispensable for answering detailed questions, finding information on rare diseases, or uncovering the latest developments [2]. Physicians trying to answer patient-related questions using PubMed during daily medical care are confronted with the difficult task of retrieving only relevant information. Retrieving a limited set of articles that is likely to answer the question requires skill. After potentially relevant articles have been retrieved, a critical appraisal must follow to determine the methodological quality of each study. Tools that help in retrieving a small set of methodologically strong trials can help the physician find relevant answers to the question.

Filters help reduce the number of articles retrieved by selecting articles based on specific characteristics. PubMed provides single term filters, available from the limits section, that select articles based on specific Medical Subject Headings (MeSH), publication types, or dates to narrow the search. Some of these limits are recommended by evidence-based search guidelines as being particularly suited to answering patient-related questions [3, 4]. Identification of reports based on methodologically sound trials, which comply with internationally accepted rules for conducting scientific trials and reporting results (Consolidated Standards of Reporting Trials [CONSORT] Statement [5]), can help reduce the number of irrelevant results. Haynes et al. developed special search filters aimed at retrieving methodologically sound trials about therapy, diagnosis, prognosis, and etiology [6-11]. To develop these filters, Haynes et al. used a set of 161 clinical journals [7] from which all articles were evaluated for relevance to the subject (therapy, diagnosis, etc.) and appraised for methodological quality according to the process delineated in Table 1.

Haynes et al. designed both sensitive and specific filters regarding therapy using this method. The sensitivity of sensitive filters, aimed at retrieving all the relevant information, and the specificity of specific filters, aimed at correctly identifying irrelevant information, are very high, suggesting that they are fully able to select methodologically strong trials regarding therapy. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.