Magazine article Technology & Learning

The Lowdown on Scientific Research and What It Means for Schools: The Idea of Imposing a Scientific Research Model on the Elementary School Reading Classroom Is Striking Fear into the Hearts of Many Educators. Here, an Expert Demystifies and Analyzes the Various Methodologies

Magazine article Technology & Learning

The Lowdown on Scientific Research and What It Means for Schools: The Idea of Imposing a Scientific Research Model on the Elementary School Reading Classroom Is Striking Fear into the Hearts of Many Educators. Here, an Expert Demystifies and Analyzes the Various Methodologies

Article excerpt

In order to qualify for funding under Reading First, all reading curricula, print- and technology-based, will need to show research-supported effectiveness.

The gold standard of such research is a "true" experiment that requires random assignment of students to either an experimental treatment group (who receive the new instructional approach) or a no-treatment control group. A pretest is given to both groups, then the experimental students receive the new treatment, then both groups take the posttest. The test score data are subjected to statistical analysis (typically an analysis of variance) to determine whether, on average, the experimental treatment students demonstrated statistically superior achievement performance than the control students.

Despite the fact that the DOE seeks scientifically based research supporting the effectiveness of reading curriculum materials at the high level of true experiments, most K-12 research does not and will not follow a true experimental design because random assignment of students to treatments is not feasible in most schools.

Practical alternatives to true experiments are "quasi-experimental" designs. Quasi-experiments are like true experiments in that they involve comparison between an experimental treatment group and a no-treatment control group. However, in quasi-experiments, student subjects are not randomly assigned. Two respected quasi-experimental designs are:

* Nonequivalent group designs, which are almost identical to true experiments, but without random assignment of subjects. Instead, intact classes of students are often randomly assigned to either the experimental treatment group or the no-treatment control group. It is highly likely that federal and state Reading First regulations and guidelines will accept this type of design as meeting the standard of scientifically based research.

* Cohort designs, which involve comparison between two groups of students from the same schools or districts who differ by age and when they received instruction. In this design, the no-treatment control group would be an older group of students who received instruction before the new treatment was implemented, and the experimental group would be a younger group of students who received instruction after the new treatment was implemented. The historical academic performance of the older group (when they were the same age/grade as the younger students were during the treatment period) is compared to the current academic performance of the younger group.

While cohort designs are generally accepted among education researchers, we haven't read nor heard anything to suggest whether federal and state Reading First regulations and guidelines will or won't accept this type of design as meeting the standard of scientifically based research.

Additional federal criteria for scientifically based research stated for the NCLB Act Title I Reading First program (Section 1208) are as follows:

(A) applies rigorous, systematic, and objective procedures to obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and

(B) includes research that

(i) employs systematic, empirical methods that draw on observation or experiment;

(ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn;

(iii) relies on measurements or observational methods that provide valid data across evaluators and observers and across multiple measurements and observations; and

(iv) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.