Academic journal article Environmental Health Perspectives

Enhancing Credibility of Chemical Safety Studies: Emerging Consensus on Key Assessment Criteria

Academic journal article Environmental Health Perspectives

Enhancing Credibility of Chemical Safety Studies: Emerging Consensus on Key Assessment Criteria

Article excerpt

OBJECTIVES: We examined the extent to which consensus exists on the criteria that should be used for assessing the credibility of a scientific work, regardless of its funding source, and explored how these criteria might be implemented.

DATA SOURCES: Three publications, all presented at a session of the 2009 annual meeting of the Society for Risk Analysis, have proposed a range of criteria for evaluating the credibility of scientific studies. At least two other similar sets of criteria have recently been proposed elsewhere.

DATA EXTRACTION/SYNTHESIS: In this article we review these criteria, highlight the commonalities among them, and integrate them into a list of 10 criteria. We also discuss issues inherent in any attempt to implement the criteria systematically.

CONCLUSIONS: Recommendations by many scientists and policy experts converge on a finite list of criteria for assessing the credibility of a scientific study without regard to funding source. These criteria should be formalized through a consensus process or a governmental initiative that includes discussion and pilot application of a system for reproducibly implementing them. Formal establishment of such a system should enable the debate regarding chemical studies to move beyond funding issues and focus on scientific merit.

KEY WORDS: chemical safety, credibility, industry funding, regulatory science, reliability, scientific integrity. Environ Health Perspect 119:757-764 (2011). doi:10.1289/ehp.1002737 [Online 15 December 2010]

Federal agencies such as the Food and Drug Administration (FDA), National Toxicology Program (NTP), National Institute of Occupational Safety and Health (NIOSH), and National Institute of Environmental Health Sciences (NIEHS) devote substantial resources to evaluating chemical hazards. However, chemical product manufacturers have conducted the great bulk of toxicological testing and research used in regulatory safety assessments. For substances whose regulation requires preapproval (e.g., pharmaceuticals, food additives, and pesticides), regulatory frameworks require companies to conduct specific tests [Federal Food, Drug, and Cosmetic Act (FFDCA) 1938; Federal Insecticide, Fungicide, and Rodenticide (FIFRA) of 1972]. The U.S. Environmental Protection Agency (EPA) also has the power, by rule, order, or negotiated outcome, to direct companies to test chemicals they manufacture or process (or propose to manufacture or process) [Toxic Substances Control Act (TSCA) 1976]. In addition, companies routinely evaluate potential hazards and risks of their products to assure a safe workplace, for product stewardship, and to limit potential liability. As a result, industry-conducted or -funded research provides the bulk of the science relevant to assessing chemical safety, and this is not likely to change (Barden et al. 2006).

For almost as long as industry has been conducting research involving chemicals, skeptics have challenged the credibility of that work. They have described intentional generation of false or misleading data (Hirschhorn 2000) and research that seems directed to increasing doubt about health effects (Michaels 2008). Critics have argued that industry-supported work has employed methods, animal strains, or other rest features that tend to miss or underestimate adverse effects (Myers et al. 2009). Commonly, no more specific criticism is leveled than that the results of the work tend to support the use, rather than restriction, of the chemical (Kissinger and Rust 2008). News and social media increasingly imply that industry support of scientific work is alone sufficient to invalidate it (Popken 2008). Even though the source of funding has been asserted to be a "less significant" cause of publication bias than other causes (e.g., academic pressure to publish) (DeMaria 2004; Fanelli 2010), industry support suffices for many to vitiate the credibility of scientific work.

The volume of industry-supported studies of chemicals is burgeoning now with the European Union's implementation of its Registration, Evaluation, Authorisation and Restriction of Chemical substances (REACH) system (Hartung and Rovida 2009), and an amended TSCA in the United States could drive additional chemical testing, even if REACH data are considered for TSCA purposes (Safe Chemicals Act of 2011). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.