Conceptual Model of Comprehensive Research Metrics for Improved Human Health and Environment
Engel-Cox, Jill A., Van Houten, Bennett, Phelps, Jerry, Rose, Shyanika W., Environmental Health Perspectives
The mission of the National Institute of Environmental Health Sciences (NIEHS) is to reduce the burden of human illness and dysfunction from environmental causes. This mission is furthered partly through funding of extramural research in science that focuses on the cellular and molecular basis of environmentally induced disease. Other types of projects funded as part of the extramural research portfolio include epidemiologic and community-based participatory research, as well as worker training and education. NIEHS is achieving its mission by focusing on diseases for which there is a strong indication of an environmental component, and for which there is high or increasing prevalence in the U.S. population (e.g., asthma); by fostering integrated research teams testing complex hypotheses that address the interplay of environmental and other factors, such as genetics, sex or gender, age, and lifestyle; and by developing initiatives identifying the complex factors in the environment that can increase the risk of disease by supporting basic research that develops the scientific basis for health decisions, as well as applied research that fills gaps in understanding of environmental health risks (NIEHS 2006b).
Given the complexity and diversity of research, program evaluation is critical to understanding and documenting the effectiveness of funded research in illuminating the linkages between the environment and human health. Mandates such as the Government Performance and Results Act of 1993 have required research agencies to look beyond measures of output (e.g., publications produced) toward metrics related to long-term outcomes on public health. Guidance from the U.S. Office of Management and Budget Program Assessment Rating Tool (PART) requires that outcomes of a program (managed by a single entity) be linked to a clear set of program and agency goals, yet be external to the research program (Office of Management and Budget 2006). When reviewing fundamental research programs using the PART guidance, managers of these programs face significant challenges in demonstrating a link between traditional research outputs and outcomes (Cozzens 1997). Health and environmental research organizations such as NIEHS have been challenged to define and measure outcomes distant in time and space from environmental health research (Van Houten et al. 2000). Outcome-based measures of accountability for research grants are inherently difficult, because by definition in the Federal Grants and Cooperative Agreement Act of 1977, grants have indirect benefit to and little substantial involvement by federal agencies.
The objective of this study was to develop a conceptual framework to measure the impact of environmental health research programs on human health, the environment, and the economy, even when the impact may be indirect or diffuse.
Describing a research portfolio as comprehensive and multidisciplinary as that of NIEHS and measuring its effect on environmental health require a strategic approach that acknowledges all of the potential components of the research process and the application of that research to society in order to ultimately improve human health and quality of life. To design this approach, we developed a comprehensive logic model describing the agency's extramural research portfolio from grant award through ultimate outcomes. Logic models are graphic depictions of the relationship between a program's activities and its intended outcomes (Centers for Disease Control and Prevention 2005; Department of Health and Human Services 2002) and help to explain a program's "theory" or the underlying structure of how the program is intended to work (Chen 2005). Besides being an evaluation tool, a logic model can also help program managers describe, and make explicit, how program "performance" is designed to achieve outcomes (McLaughlin and Jordan 1999). Research programs have extended traditional program logic to illustrate how research contributes to topics that inform federal decisions about protective health standards (e. …