Interpreting Outcomes: Using Focus Groups in Evaluation Research*
Ansay, Sylvia J., Perkins, Daniel F., Nelson, John, Family Relations
Although focus groups continue to gain popularity in marketing and social science research, their use in program evaluation has been limited. Here we demonstrate how focus groups can benefit evaluators, program staff, policy makers and administrators by providing an in-depth understanding of program effectiveness from the perspective of participants as stakeholders in program outcomes. Using data from the Youth Action Program, a prevention program funded by the United States Air Force for military dependent adolescents at risk, our methodology included a narrative approach to data analysis, combining theme analysis with a systematic interpretive approach centering on how group members experienced the program within the contexts of their daily lives. We show that the parents benefited from the program beyond youth outcomes, and this was useful information for program improvement and expansion.
Key Words: adolescents, evaluation, focus groups, military dependents, narrative analysis, youth at risk.
(Family Relations, 2004, 53, 310-316)
Despite their widespread use in marketing research since the 1950s, focus groups have a relatively short history of use in social science research. During the 1990s, their popularity increased so rapidly that Morgan (1997) noted their appearance in 100 academic journal articles per year throughout the decade in his review of online databases. He provided examples of focus groups being used as a principal data source, as supplementary to survey data, and in multimethod studies combining other methods. The current popularity of focus groups in the social sciences appears to stem from their adaptability, as they are linked to interviewing, participant observation, and survey research.
The use of focus groups in social program evaluation has been slower to gain popularity. One explanation for this lag may lie in the emphasis on individual outcomes approaches that typically highlight social program evaluation (Jacobs, 1988). For example, the Urban Institute (Eisen, Pallitto, Bradner, & Bolshun, 2000) profiled 51 promising prevention programs and approaches for at-risk youth. Each program appears to rely on the use of traditional scientific methods of random sampling, comparison or control groups, and surveys or other quantitative methods, with statistical significance as the sole measure of effectiveness or success. Individual outcomes (i.e., the summative effect of the programs on youth) are the sole indicators of program success used in these ratings.
Whereas the traditional scientific model provides valuable and unique data and generates a broad, generalizable set of findings (Patton, 1987), it is limited in its representation of programs as static and isolated from the broader social milieu in which they operate. Moreover, the multifaceted designs of many intervention studies "[make it] difficult to disentangle the 'active ingredients' of a programme" (Farrington & Welsh, 1999, p. 291). Thus, Patton (1997) suggested that evaluation research move in another direction, one that views evaluation as part of the overall process of the program itself. Labeled utilization-focused research, he stated:
Departing from defining evaluation as rigorous application of social science methods opens a different direction in evaluation , one that supports integration of evaluation into program processes. Making data integral rather than separate can reinforce and strengthen program intervention, (p. 93)
His suggestion is being implemented in program development. A five-stage model suggested for use in developing prevention programs for high-risk families (Dumka, Roosa, Michaels, & Suh, 1995) describes how focus groups aided researchers in identifying a set of mediator variables that were used to define the problem, select culturally sensitive program objectives, and provide data for formative evaluation of a pilot program. The model's designers noted the particular strength of qualitative methods, including focus groups, "in adapting program content to the conditions, beliefs, and language of the local community" (p. …