Evaluation of Project P.A.T.H.S. (Secondary 1 Program) by the Program Participants: Findings Based on the Full Implementation Phase
Shek, Daniel T. L., Sun, Rachel C. F., Adolescence
To promote holistic development among adolescents in Hong Kong, the Hong Kong Jockey Club Charities Trust has approved HK$400 million to launch a project entitled "P.A.T.H.S. to Adulthood: A Jockey Club Youth Enhancement Scheme" (Shek, 2006a). There are two tiers of programs (Tier 1 and Tier 2) in the P.A.T.H.S. Project. The Tier 1 Program is a universal positive youth development program in which students in Secondary I to Secondary 3 will participate, normally with 20 hours of training in the school year at each grade involving 40 teaching units developed with reference to 15 positive youth development constructs (Shek, 2006b, 2006c; Shek & Ma, 2006). There are two implementation phases in this project--Experimental Implementation Phase and Full Implementation Phase. For the Experimental Implementation Phase, 52 secondary schools participated with the objectives of accumulating experience in program implementation and familiarizing frontline workers with the program design and philosophy. Following the Experimental Implementation Phase, 207 schools joined the Secondary 1 Program in the Full Implementation Phase.
Based on the principle of triangulation, evaluation data based on different strategies and sources have been collected in order to determine the program's effectiveness and effect. These include an objective outcome evaluation utilizing a randomized group trial (Shek, Siu, Lee, Cheung, & Chung, 2008), a subjective outcome evaluation based on quantitative and qualitative data collected from the program participants and instructors (Shek & Ma, 2007; Shek, Siu, & Lee, 2007; Shek & Sun, 2007a, 2007b), process evaluations involving systematic observations of delivery of the program (Shek, Lee, & Sun, 2008; Shek, Ma, Lui, & Lung, 2006; Shek, Ma, Sun, & Lung, 2008), interim evaluation (Shek, Ma, & Sun, 2008; Shek & Sun, 2006; Shek, Sun, & Siu, 2008), and qualitative evaluation, including focus groups involving students and program implementers (Shek & Lee, in press; Shek, Lee, Siu, & Lam, 2006), case study (Shek & Sun, in press-a), in-depth interviews with program implementers, student logs, and student products (Shek, Sun, Lam, Lung, & Lo, 2008).
With specific reference to subjective outcome evaluation, quantitative findings based on the Experimental Implementation Phase showed that program participants (N = 8,679 students from 52 schools) and instructors (N = 344) had positive perceptions of the program and instructors. In addition, roughly four-fifths of the participants and instructors perceived the program to be beneficial to the participants (Shek & Ma, 2007; Shek, Siu, & Lee, 2007). Similarly, qualitative findings derived from subjective outcome evaluation showed that the program was perceived positively by the participants and implementers and was regarded as beneficial to them (Shek & Sun, 2007a, 2007b). Furthermore, findings showed that quantitative subjective outcome evaluation findings converged with objective outcome evaluation findings (Shek, Lee, Siu, & Ma, 2007; Shek & Sun, in press-b).
Although the existing subjectives outcome evaluation findings are encouraging, they were based on 52 schools in the Experimental Implementation Phase only. Since replication is an important process in determining whether patterns observed in one study can be repeated in another, in the present study, the data collected from 207 schools participating in the Full Implementation Phase were examined. As Project P.A.T.H.S. was financially supported by the Hong Kong Jockey Club Charities Trust, each participating school was required to submit an evaluation report with the consolidated subjective outcome evaluation profile of the school to the funding body. In other words, because the workers were expected to conduct a program evaluation as part of their professional practice, we could make use of such reports to "reconstruct" the overall profile of the subjective outcome evaluation data. …