The Evaluation of Leisure Programs: Applying Qualitative Methods
Howe, Christine Z., JOPERD--The Journal of Physical Education, Recreation & Dance
As the challenges of society become increasingly costly and complex for leisure professionals, the importance of responsible decision making in the use of resources is paramount. Conscientious leisure programmers faced with diverse recreational demands from competing and sometimes conflicting groups must make decisions as fairly as possible. What can programmers use to help them make these decisions?
Evaluation, as one of the crucial steps in leisure programming, can help programmers make better decisions (Edginton, Hanson, & Edginton, 1992; Farrell & Lundegren, 1988; Howe & Carpenter, 1985; Rossman, 1989; Russell, 1982). Evaluation is usually defined as a systematic process of judging the merit of a program for either formative or summative purposes. Evaluation requires collecting information to serve as the basis for planning, analysis, and decision making. The process of systematic evaluation is similar to conducting applied research (Kraus & Allen, 1987; Lundegren & Farrell, 1985; Theobald, 1979, 1985). So, the methods that are available to use in applied research can also be used in program evaluation.
What determines the value of a program? Two broad indicators of a program's value include efficiency and effectiveness. If a program is efficient, it uses resources prudently and is worthy of continued investment. If a program is effective, it is enjoyable, meaningful, and satisfying for the participants. A programmer's determination of specific evaluation criteria directs his or her selection of evaluation method. When a programmer focuses on efficiency, a rationalistic evaluation approach is often taken using quantitative methods to collect and analyze numerical data (e.g., pricing, participation rates). When a programmer focuses on effectiveness, a naturalistic evaluation approach is useful. Qualitative methods to collect and analyze open ended written, spoken, or observed information can be used (e.g., What does participating in the low impact aerobics program mean to you? Why did you discontinue attending the stamp collecting club?).
This article discusses the effectiveness component of leisure program evaluation. First, the naturalistic program evaluation approach is introduced. Then, the qualitative information collection and analysis technique of participant observation is highlighted. Observation, interviews, the review of documents, and content analysis are described. Finally, some suggested applications to professional practice are offered.
Naturalistic Program Evaluation
Before beginning any program evaluation, to learn something about context, the philosophy, goals, and objectives of the leisure service organization should be reviewed. This background can help build the foundation for the first few evaluation questions or issues that programmers will use to access the participants, guide initial observations, or start interviews.
When focusing on effectiveness, programmers want to know why participating in the program facilitates meaningful, fulfilling leisure expression as perceived or judged by participants. In naturalistic evaluation, the information gleaned from participants through observing their behavior over time, indepth interviewing, or examining items they have created (drawings, stories, etc.) is key to evaluating the program. The judgment criteria or evaluation questions concentrate on participants' enjoyment, meaning, and satisfaction (Rossman, 1980, 1989). Once the first few evaluation criteria/questions are established, programmers must determine how to best gather evidence from participants. This leads to the evaluation design - the interrelated information collection, analysis, and interpretation process.
Qualitative Information Collection and
Analysis via Participant Observation
The collection, analysis, and interpretation of qualitative information is an inquiry process that follows a general game plan. …