Academic journal article Education Next

The Virtues of Randomness: For Years, Researchers Have Used Randomized Field Trials to Evaluate the Effectiveness of Medical Procedures. (Feature)

Academic journal article Education Next

The Virtues of Randomness: For Years, Researchers Have Used Randomized Field Trials to Evaluate the Effectiveness of Medical Procedures. (Feature)

Article excerpt

Educators are finally starting to catch on

The principle that social interventions ought to be evaluated has a long pedigree. Eager readers of the Muquadimah know that Ibn Khaldun considered competing explanations for the success of Arab regimes in the 13th century. In the 19th century, Florence Nightingale reproved the English Parliament for failing to weigh seriously the consequences of making changes to social programs, charging that "without an inquiry after results, past or present, it is all experiment, see-saw, doctrinaire, a shuttlecock between two battledores." A 21st-century Nightingale could testify at a congressional hearing with only slight revisions. After a 1995 review of evidence on the performance of federal employment and training programs, the General Accounting Office concluded that "most federal agencies do not know if their programs are working."

Nevertheless, the 20th century brought great progress in the theory and methods of evaluation as well as more understanding of its necessity. The most promising innovation was the randomized field trial. Such experiments randomly "treat" individuals and even whole institutions, such as hospitals or schools, with different interventions in order to learn which work better. The random allocation ensures that the two groups being compared are not different in ways that would influence their response to a particular treatment, For instance, suppose you wanted to test the quality of a new reading curriculum. You might select some students to be taught using the new curriculum and compare their progress with that of students who stayed with the old curriculum. But the students chosen to receive the new curriculum might differ in a way that influences their academic Progress--they might, for instance, have had better reading teachers in the past, be more motivated, or have access to more educational resources at ho me, Randomly allocating students or multiple classrooms or multiple schools to the new reading program and to control conditions or to an alternative program eliminates the possibility that the two groups will differ in a systematic way and thus compromise the results.

Randomized field trials are a sturdy method of generating defensible evidence about the relative effectiveness of various interventions; nonrandomized trials do so, at times, but unpredictably. For instance, in the Salk polio vaccine studies of the 1950s, randomized trials that were mounted in some states produced estimates of the vaccine's effect on polio that were appreciably greater than estimates from a parallel series of nonrandomized trials, The factors that might have led to this difference are still not well understood. After the Salk trials, controversy erupted over the use of oxygen-enrichment therapy for premature infants, Early domized studies suggested that the infant death rate was reduced significantly by the oxygen-enrichment therapy. Subsequent randomized clinical trials helped to reveal that enriched oxygen environments for premature infants caused blindness and did not decrease infant mortality.

Yet this powerful technique of discovering what works has been slow to come to the field of education. Consider the widespread adoption of "whole school reform" models such as "Success for All," "Accelerated Schools;' and "Expeditionary Learning' The promise of whole-school reform was to install standardized, high-performing, research-tested curricula in school after school, instead of forcing each school and school district to be its own curriculum developer. Much energy and expense has been devoted to implementing these prepackaged school-reform programs in mainly low-income school districts, Nevertheless, in 1999 the American Institutes of Research reviewed all the studies done on 24 reform models and found that 5 of the programs had no evidence beyond anecdote and personal testimonials to support their claims of raising achievement, The other 19 programs were subject to about 116 independent studies of their effectiveness. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.