Mega-Analysis of Meta-Analyses
Forness, Steven R., Kavale, Kenneth A., Blum, Ilaina M., Lloyd, John W., Teaching Exceptional Children
What does the research say about: Ritalin
Social skills instruction
Special class placement
Conflicting reports on the effectiveness of these approaches abound-but meta-analysis can help sort out overall benefits and weaknesses of these and other educational practices. This article reports on some of these broad studies of research that seem to indicate effectiveand noneffective-teaching strategies.
But as special educators all know, the most important consideration is a knowledge of what works with your individual student, as well as a compelling curiosity that leads you to read the meta-analyses for yourself.
In meta-analysis, a researcher examines many studies on a particular strategy and derives a numerical indicator of the relative effectiveness of the strategy, averaged across all studies. This indicator is called an effect size (ES) (see box, "How Meta-analysis Works"). Such an analysis of research not only provides a numerical indicator of the relative effect of a particular intervention, but the effect size also allows comparison with other approaches used in special education or related services.
Available to Date
We have summarized the results of 18 meta-analyses on special education or its related services that have been done to date (but see our cautions about how some of these studies are "dated"). We also depict these studies in relation to one another so you can see which strategies are most (or least) effective. This "megaanalysis" provides a summary of the relative power of a variety of special education and related interventions, at least those to which meta-analysis has been applied thus far.
Table 1 shows the overall mean effect sizes and number of studies used for each of these 18 meta-analyses. The table shows various strategies and instructional approaches in rank order, from most to least effective, as found by the metaanalyses.
We begin at the bottom of this table with interventions in which meta-analysis produced ineffective or very modest effect sizes. We then describe those with relatively good ESs and finish with those at the top of the table, in which effect sizes were quite convincing as to the effectiveness of the interventions in question. In this summary, we also point out certain ESs within each meta-analysis that suggest circumstances in which the intervention seems to work best.
Interventions with Modest Eff Sizes
In this table, the bottom five interventions are those in which meta-analysis produced relatively modest ESs. As a matter of fact, the 50 efficacy studies on special class placement produced a negative ES, suggesting a potentially harmful effect. It should be noted, however, that most ESs in these meta-analyses were from studies on special class placement of children with mild mental retardation. In those few studies on placement of children with learning disabilities or emotional and behavioral disorders, the mean ES was .29, suggesting an improvement of 11 percentile ranks (i.e., an unplaced child with learning disabilities or behavioral disorders would be at the 50th percentile, and an ES of .29 would move that child to the 61st percentile). It is also interesting to note that the overall ES of -.12 was equivalent to a loss of only about 1 or 2 months in academic achievement during an average special class placement of about 2 years. Although this meta-analysis was done 17 years ago (1980), relatively few well-controlled efficacy studies have been conducted since then.
Both perceptual-motor training and its counterpart, in which children with socalled visual-perceptual deficits are placed in linguistic-based interventions or vice-versa (modality instruction or learning styles) produced negligible ESs. …