Measuring Partnership Activities: Partnerships in Environmental Public Health Evaluation Metrics Manual
Drew, Christina H., Pettibone, Kristianna G., O'Fllon, Liam R., Collman, Gwen W., Birnbaum, Linda S., Environmental Health Perspectives
The National Institute of Environmental Health Sciences (NIEHS) has had a long-standing commitment to facilitate and engage community groups in environmental health science research. In 2008, the NIEHS established the Partnerships for Environmental Public Health (PEPH) program to formalize our commitment to outline a coordinated vision for community and academic partnerships. Since then, > 400 grantees have participated in activities designed to foster networking among grantees within the various NIEHS programs, including webinars and workshops on communicating PEPH findings and translating research to public health policy.
A key tenet of the PEPH is community engagement. In response to an NIEHS Request for Information in 2008, the community shared concerns about the lack of evaluation capacity and the need for tools and approaches to develop project specific evaluation metrics for public health-related program areas. In response, the NIEHS developed the PEPH Evaluation Metrics Manual (NIEHS 2012b) with significant input from PEPH grantees, program staff; and experts in the field, including input from > 250 individuals at > 30 professional meetings.
Evaluation of PEPH programs provides useful benefits to grantees, including the ability to a) identify program successes; b) determine whether a project worked and why (or why not); c) identify areas for program improvement and increased efficiency; d) describe expenditures and justify a need for additional funding; e) recognize and respond to public needs and wants; f) identify new audiences and applications for projects; and g) prioritize research and plan for the future. Evaluation also may help grantees find allies in other agencies, services, or sectors; publicize achievements in communities; and inform policy and other decision making. Evaluation metrics also provide a means for the NIEHS to evaluate the success of individual projects and the PEPH program as a whole.
Typical approaches to evaluating research outcomes involve analyzing publications. However, because many PEPH programs do not publish findings related to their community engagement, we worked with grantees and community members to identify appropriate metrics to measure and demonstrate success in five areas that are common to many PEPH grantees:
* Partnering (working with other organizations to conduct environmental public health activities)
* Leveraging (using the resources already available to a project to obtain additional resources)
* Disseminating findings (providing information about environmental public health issues and results of PEPH research)
* Training (developing programs that teach researchers, community members, workers, students, and others strategies for reducing hazardous environmental exposures)
* Capacity building (performing activities that improve an organization's ability to achieve its mission).
For each of these five areas, the NIEHS developed an illustrative logic model to demonstrate connections among project activities, outputs, and impacts; > 80 examples of metrics for each activity, output, and impact are provided as examples for grantees developing metrics for evaluating the progress and achievements of their own programs.
We present the partnership logic model (Figure 1; NIEHS 2012b) to illustrate how metrics can be developed from a logic model. When developing program logic models, it can be helpful to begin by working with partners to identify and articulate the desired impacts of the program. Once partners agree about "where they want to go" (impacts), discussions of "how to get there" (activities) naturally follow. Then partners can determine "how will we know we are there?" (outputs and metrics). The nouns, adjectives, verbs, and adverbs in the answers to these questions then serve as the basis for metrics.
Examples of metrics from grantee programs that address components of the partnership logic model include:
* Demonstrating success at identifying partners (activity 1), the University of Cincinnati's anti-idling campaign provided a description of the partners involved and the resources they bring to the project: Cincinnati Public Schools provided access to students and schools; the Cincinnati Health Department provided nursing services; a councilwoman provided credibility and the ability to attract attention to the project; and the Hamilton County Department of Environmental Services provided training and information to Cincinnati Public School staff and students. …