An Evaluation of the Ontario Rapid Risk Factor Surveillance System
Bingle, Catherine L., Holowaty, Philippa H., Koren, Irene E., Picard, Louise, et al., Canadian Journal of Public Health
Background: The Rapid Risk Factor Surveillance System (RRFSS) is an ongoing population health survey conducted by a collaborating group of Ontario public health units. This formative evaluation examined the process effectiveness, collaboration, utility and cost-effectiveness of RRFSS during its first year of operation.
Methods: An Evaluation Framework was developed with reference to guidelines for evaluation of surveillance systems developed by the World Health Organization and the U.S. Centers for Disease Control and Prevention. The study focussed on evaluable performance areas in a young surveillance system and on information needed to inform stakeholder decisions about future participation and improvement. Data were collected through surveys and interviews of key informants in participating health units, non-participating health units, the survey research house, and the provincial health ministry.
Results: Findings documented early use and dissemination of RRFSS data in health units after less than a year of surveillance system operation, stakeholder perceptions overall of data impact and value, and satisfaction with system functioning. Challenges to effectiveness were documented concerning data analysis, barriers to data use, and sustainability. Performance improvement strategies were identified for survey implementation and supports, data use, system participation, and reduced costs.
Conclusion: In its first year, RRFSS was an effective collaborative method to collect population data for public health program planning and evaluation. The evaluation provided valuable information on use, functioning, effectiveness, strategic issues and areas for improvement in a young surveillance system, created opportunities for stakeholder input into evaluation and planning, and provided a baseline for future evaluations.
MeSH terms: public health, population surveillance, evaluation studies, program evaluation, data collection, cooperative behavior
The Rapid Risk Factor Surveillance System (RRFSS) is a population health survey conducted by a collaborating group of Ontario public health units, and administered by the Institute for Social Research (ISR) at York University. Each month, random samples of 100 adult residents (aged 18 and older) in each of the health unit districts are surveyed by telephone in either English or French. The purpose of RRFSS is to provide data that are timely, relevant to local public health needs, useful for surveillance, and responsive to emerging public health issues.1 It is designed to minimize the need for one-time surveys by individual health units and maximize overall public health efficiency through a coordinated system.2 This article describes the methodology, findings, and subsequent actions from a formative evaluation of RRFSS.3
The framework for RRFSS was based on the Behavioral Risk Factor Surveillance System (BRFSS) used in the United States. RRFSS was developed in 1999 as a pilot in Durham Region, Ontario, by a partnership among Health Canada, the Ontario Ministry of Health, Cancer Care Ontario, and the Durham Region Health Department. Following the success of the pilot,4 in 2000 with financial support from the Ontario Ministry of Health and Long-Term care (MOHLTC), a group of three health units (Durham Region, Simcoe County, and Haliburton-Kawartha-Pine Ridge District) developed a plan for expanding RRFSS, revised the questionnaire, and developed a data dictionary. Through consultations with health unit staff, representatives from the MOHLTC, and local health and social services agencies, the group identified needs for information suitable to the RRFSS format. In addition, provincial guidelines for public health unit programs and key data sources such as the Canadian Community Health Survey (CCHS), were reviewed to identify data gaps that could be addressed by RRFSS. When there were no pre-existing suitable questions, questions were developed with the help of ISR, health unit staff, MOHLTC consultants and experts in the field. …