Theoretical Soundness, Proven Effectiveness, and Implementation Fidelity of the HOSTS Language Arts Program among Children Identified as At-Risk in Urban Elementary Schools
Senesac, Barbara V., Burns, Matthew K., Journal of Instructional Psychology
In order to fully evaluate the quality of evidence for any educational innovation, research is needed regarding consistency with theory, demonstrated effectiveness, and consistent implementation. The Help One Student to Succeed (HOSTS) Language Arts program was specifically mentioned in the No Child Left Behind act as a program that incorporates community involvement to improve student reading skills. However, few published studies exist with which the quality of evidence could be evaluated. The current study examined existing research regarding theoretic soundness and demonstrated effectiveness, and conducted a study to examine the large-scale fidelity of implementation. Participants consisted of 51 elementary schools in urban areas, and 1,354 students. Results suggested that a high level of fidelity was obtained significantly more frequently than could be expected, and students in HOSTS programs that were implemented in a consistent manner achieved better reading outcomes. Potential implications for practice are included.
The No Child Left Behind Act (NCLB) mandated research-based educational practices in American schools. As a result, there seems to be an increased interest in high-quality research driving practical educational decision making (Berends & Garet, 2002; Eisenhart & Towne, 2003; Slavin, 2002). However, Berends and Garet (2002) claimed that informed educational policy debates require data from surveys or assessments from a representative sample of schools. Thus, it could be argued that both randomized or quasi-experimental and applied studies are needed to fully examine the utility of an educational program.
Ellis (2001) reviews research on several educational innovations and provides a model with which research can be used to evaluate an innovation. Educational programs, according to Ellis, should be supported by research on three levels. Level I research is basic research, which leads to a sound theoretical base, or examines the consistency of an innovation with a given theory. Level II research is conducted in applied settings and examines the efficacy of the educational program in improving the education of children. Randomized controlled research could occur in either Level I or II, with controlled settings such as a laboratory being appropriate for Level I and applied settings such as schools being Level II. Level II research is also consistent with the NCLB call for data from quasi-experimental studies. Finally, Level III research examines the effectiveness of the innovation after wide-scale implementation. Innovations implemented on a large scale, but that lacked a sound theoretical base and/or researched effectiveness, could be considered an educational fad (Ellis, 2001).
In addition to mandated evidence-based practices NCLB (2001) advocated structured programs or learning systems that incorporate parent and community involvement to assist students in improving their reading skills and overall school performance. Singled out as a model of such a program is the Help One Student to Succeed (HOSTS) Language Arts Program; the only program specifically cited in the NCLB Conference Report (2001). The HOSTS Language Arts Program is a structured technology-based learning system designed to supplement the school's literacy curriculum and is delivered by community volunteer tutor/mentors. With the NCLB requirement of making average yearly progress on state standards, some failing schools are implementing the HOSTS Language Arts Program for supplemental educational services provided for under Title I funding.
Given that the HOSTS Language Arts Program is being used to as a supplemental service and is mentioned in the NCLB report (2001), it seems necessary to examine the evidence-base for this program. The purpose of the current article is to describe the research from Levels I and II (Ellis, 2001) for HOSTS Language Arts Program and to discuss a study which examined data from a state-wide implementation of the HOSTS program. …