The Effect of Performance Feedback on Teachers' Treatment Integrity: A Meta-Analysis of the Single-Case Literature
Solomon, Benjamin G., Klein, Suzanne A., Politylo, Bethany C., School Psychology Review
Performance feedback (PF) is defined in the school consultation literature as "monitoring a behavior that is the focus of concern and providing feedback to the individual regarding that behavior" (Noell et al., 2005, p. 88). Feedback, as a more general construct, has been studied in education extensively (Hattie & Timperley, 2007; Kulhavy & Wager, 1993). However, the use of feedback in the manner typically used by school consultants emerged out of the organizational psychology literature (Alvero, Bucklin, & Austin, 2001; Balcazar, Hopkins, & Suarez, 1985). In the school consultation literature, PF is typically used to increase the treatment integrity (TI) of a prescribed intervention across a continuum of tiered interventions for both behavioral and academic problems. Treatment integrity refers to the degree to which teachers implement an intervention or treatment accurately (Noell, Witt, LaFleur, Mortenson, Ranier, & LeVille, 2000). Although PF studies have proliferated within the consultation literature, there has yet to be a quantitative synthesis of these articles that compares the varying elements of PF and its effect on different subgroups and across different settings. Such a synthesis is important for both practitioners and researchers to be able to access so that the evidence base can inform the tailoring of PF for specific interventions and contexts. In addition, a quantitative synthesis of the literature allows for generalizable conclusions across studies.
Performance Feedback to Improve Treatment Integrity
Gresham and Kendall (1987) reviewed consultation studies published prior to 1987 and found that no study reviewed included TI data. Gresham (1989) reflected on his previous finding and concluded that most studies relied on the "consult and hope" (p. 48) approach, described as the act of consultation services occurring, but with no follow-up to ensure that teachers performed the treatment as prescribed. Gresham (1989) suggested the following reasons as to why treatments are not carried out as intended: (a) the complexity of the plan, (b) the number of treatment agents, (c) the time required to implement, (d) the resources required, and (e) perceived effectiveness or motivation of treatment agent. Many PF studies, including Duhon, Mesmer, Gregerson, and Witt (2009), Noell, Witt, Gil bertson, Ranier, and Freeland (1997), Witt, Noell, LaFleur, and Mortenson (1997), and Noell, Duhon, Gatti, and Connell (2002), have documented notable negative slopes in integrity during the baseline phase that would have gone unnoticed had follow-up not occurred. To combat threats to integrity, Gresham suggested that researchers and practitioners use direct observation coupled with the provision of behavior-specific feedback to the consultee. He also suggested self-monitoring strategies. Performance feedback is a tool to manipulate levels of TI so practitioners can conclusively state whether students received the intended intervention or support.
Key Features of School-Based Performance Feedback
Performance feedback has been analyzed in a variety of contexts to increase the TI of a broad array of classroom-based interventions. Furthermore, PF has been modified in different ways to suit the unique needs of the consultant-consultee dyad. The key features described in the following are clearly identified in the published literature and individual studies can be reliably grouped along these dimensions. Explanation of other features of PF is provided in the discussion section.
Performance feedback has been used to increase the integrity for a variety of interventions, such as plans aimed at increasing on-task behavior for individual students, small groups of students, and whole class interventions with strategic use of behavior-specific praise, discrete trial instruction, redirections, and a variety of reinforcement schedules (e.g., Codding, Livanis, Pace, & Vaca, 2008; DiGennaro, Martens, & McIntyre, 2005; Martens, Hiralall, & Bradley, 1997; Myers, Simonsen, & Sugai, 2011; Noell et al. …