Academic journal article Exceptional Children

Quality Indicators for Group Experimental and Quasi-Experimental Research in Special Education

Academic journal article Exceptional Children

Quality Indicators for Group Experimental and Quasi-Experimental Research in Special Education

Article excerpt

This article presents a set of quality indicators for experimental and quasi-experimental studies for the field of special education. We believe there is a critical need for such a set of indicators, given the current focus on the need for an increase in rigorous, scientific research in education. Recently, the National Research Council (NRC, 2002), in a report on scientific research in education, noted that they saw no reason why education could not be subject to the same scientific methods as other disciplines such as chemistry or physics. They further argued that there is a place for all research methodologies in educational research: survey research, qualitative research, and correlational research. As Feuer, Towne, and Shavelson (2002) noted, "If a research conjecture or hypothesis can withstand scrutiny by multiple methods, its credibility is enhanced greatly. Overzealous adherence to the use of any given research design flies in the face of this fundamental principle" (p. 8).

Yet the NRC report is unequivocal in stating numerous times that experiments using randomized trials are currently underutilized in educational research, despite being "the single best methodological route to ferreting out systematic relations between actions and outcome" (Feuer et al., 2002, p. 8). In that sense, they reiterate a point made over a generation ago by Campbell and Stanley (1966), who highlighted that controlled experimental research conducted in field settings was

   the ... only means for settling disputes regarding
   educational practice, as the only way of verifying
   educational improvements, and as the only way
   of establishing a cumulative tradition in which
   improvement can be introduced without the
   danger of a faddish discard of old wisdom in
   favor of inferior novelties. (p. 2)

Until recently, there was no specific set of standards or quality indicators for evaluating the quality of either proposed or completed experimental or quasi-experimental intervention research. Few of the numerous educational research textbooks provide a clear list of criteria that fit the realities of contemporary field research, although they present excellent ideas (e.g., Boruch, 1997; Gall, Borg, & Gall, 2002; Shadish, Cook, & Campbell, 2002). Fortuitously, slightly before we began this endeavor, the What Works Clearinghouse of the U.S. Department of Education released the Study Design and Implementation Assessment Device (DIAD; Valentine & Cooper, 2003). Its major goal is to evaluate whether a research article or research report can be considered sufficiently valid and reliable to be entered into a research synthesis on the effectiveness of an intervention or approach.

Our goal is a bit broader. We intend these quality indicators to be used not only to evaluate the merits of a completed research report or article, but also to evaluate research proposals, dissertation proposals, and grant applications submitted to funding agencies. We also intend this to be useful to researchers as they think through the design of a study, to serve as a checklist or organizer of critical issues for consideration.

We intentionally chose to look at quality indicators, rather than a set of "yes/no" standards, because evaluating a research design always entails weighing the relative strengths and weaknesses in a set of domains. Our goal is not to provide a basic primer on how to design a high-quality field research study. There are numerous introductory (e.g., Gall et al., 2002) and advanced texts (e.g., Shadish et al., 2002) widely available to explain the many concepts underlying the design of experimental studies. Our hope is that our set of indicators will be field-tested and refined, and then considered useful by journal editors and reviewers of federal grant proposals. We also envision this set of indicators assisting researchers as they design studies and guiding practitioners as they consider alternative educational practices for adoption in their classrooms and schools. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.