Academic journal article Journal of School Health

"Congratulations, You Have Been Randomized into the Control Group!(?)": Issues to Consider When Recruiting Schools for Matched-Pair Randomized Control Trials of Prevention Programs

Academic journal article Journal of School Health

"Congratulations, You Have Been Randomized into the Control Group!(?)": Issues to Consider When Recruiting Schools for Matched-Pair Randomized Control Trials of Prevention Programs

Article excerpt

Comprehensive school-based prevention programs that address a broad range of student outcomes, such as academic achievement, prosocial behaviors, and self-esteem, are usually applied school wide. (1) Research designs involving random assignment to conditions are necessary to provide the strongest empirical evidence concerning the effectiveness of school-based prevention programs and have been proposed as 1 criterion for a prevention program to be considered "evidence based" or efficacious. (1-10) Randomization reduces the chance that extraneous factors serve as alternate explanations for observed differences in outcomes between treatment and nontreatment schools and strengthens the claim that the intervention itself produced these differences. (11-14) For example, randomization reduces the likelihood that participant selection bias is a factor in the outcome and thus strengthens the internal validity of the results. (14)

If an intervention is applied school wide, it is best to randomize at the school level for evaluation purposes. Randomization at the classroom or student level within a school may contaminate program effects because students typically interact with each other during the school day. (6) Because students are clustered within schools, analyses can then be adjusted to take into account any nonindependence that this introduces into data associated with different students in the same school (as indexed by the intraclass correlation coefficient). (15) Because the number of schools being randomized is typically relatively small in trials of school-wide interventions, randomization alone typically will not ensure equivalence between treatment and control conditions. (6,16) To address this concern, the matched-pair randomized control trial (MP-RCT) may be used. In this design, schools are matched into pairs based on relevant variables, such as standardized academic test scores or number of total students enrolled, and then 1 member of each pair is randomized into a control or treatment group. (11,14) The matched-pair element increases the chance that the conditions of the control and treatment schools are similar on measured and nonmeasured factors and that results will not be attributable to preexisting differences. (6,7,11,13,14)

Recruiting schools for an MP-RCT presents unique challenges for researchers. (17) An assumption that schools will simply acquiesce to their randomly assigned condition has the potential to backfire. Schools may not want to participate in an MP-RCT, for example, because they may prefer to self-select their conditions. (18) In this regard, some schools may prefer to be in the treatment condition because they desire a program that promises to bring immediate benefits to their students. (13,19) Kam et al found that they could not randomly assign schools to test an intervention because the local community required that schools with students who were considered to be most "at risk" be given priority in receiving the intervention. (20) Conversely, some schools may prefer to be in the control condition because they are overwhelmed with too many other demands on staff time or they doubt the efficacy of the new program. (13,19)

If schools decline participation, the sample's representativeness, and hence the study's external validity, may be compromised because the participating schools are different from the nonparticipating schools. An overall reduction in the number of participating schools, furthermore, decreases the statistical power of the study to detect the intervention's effectiveness. This issue is an especially important concern for MP-RCTs because in clustered statistical analyses, statistical power will be dependent on the number of schools more so than the number of students sampled at each school. (15) Fewer available schools also may make it more difficult to match schools into pairs, potentially affecting both internal validity and statistical power.

The possibility that a school could refuse participation in an MP-RCT because the school did not want to be randomized into a condition has implications for researchers' recruiting strategies. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.