A Teacher's Guide to Meta-Analysis

Article excerpt

Ms. Quinn, the special education director, convened her monthly meeting with the district's special education teachers. The theme of Ms. Quinn's discussion was the implementation of evidence-based practices required by the No Child Left Behind Act. During this discussion she brought up the term meta-analysis in reference to research studies, stressing that this type of literature review may be helpful when selecting effective practices for students with special needs. The term was a new one for many of the teachers and even the ones who were aware of the term were perplexed how meta-analysis could possibly impact their day-to-day classroom teaching.

Technical terms such as meta-analysis, research synthesis, and effect size may sound alien to practitioners in the field of education. Due to limited knowledge on meta-analysis among educators, the results of meta-analyses seldom translate from research to classroom practice-even though metaanalyses may provide valuable information on how to select effective evidence-based practices and strategies. It is therefore essential to build teachers' knowledge base to enable them to critically evaluate meta-analyses and use the findings to inform their instructional practices. Increasing teachers' utilization and understanding of metaanalytic results may in turn result in positive outcomes for persons with disabilities in the areas of education, employment, and beyond.

What Is Meta-Analysis?

In the 1970s, Gene Glass first introduced the concept of meta-analysis, which attempts to statistically accumulate research findings from individual studies into a review summary or combine those results across studies (Hedges, 1986; Kavale, 1984, 2001). Meta-analysis originated in the fields of agricultural science and medicine, and later became a popular method in psychology and education to integrate research studies. Meta-analysis results can summarize a large body of instruction and can have a positive impact on theory and practice of special education (Guskin, 1984; Kavale, 2001); there is a growing emphasis on using such results to guide teaching and research. There have been several meta-analyses of reading, writing, and math interventions in the fields of learning disabilities, intellectual disabilities, emotional and behavioral disorders, autism, and attention deficit hyperactivity disorder, among others (see Table 1 for examples).

In their meta-analysis of intervention efficacy for students with learning disabilities, Swanson and Hoskyn (1998) found that strategy and direct instruction yielded the highest effect sizes. Three instructional components (controlling the difficulty of the task, small interactive student groupings, and direct response/questioning) were important in predicting effect sizes. In the area of intellectual disabilities, Browder and Xin (1998) found that sight-word reading instruction was highly effective. Results of metaanalysis in special education literature have identified instructional approaches (e.g., strategy and direct instruction) and specific instructional components (e.g., use of a representation when teaching math word problems) associated with increased achievement for students with disabilities (Swanson & Hoskyn; Xin & Jitendra, 1999).

How Does Meta-Analysis Differ From Traditional Narrative Review?

Education research reviews are summarized using various methods including traditional narrative reviews and metaanalysis. Narrative reviews are qualitative in nature and typically provide summaries of several studies in a particular field (Kavale & Glass, 1981). In the narrative review, the researcher may attempt to aggregate study results by reporting the number of studies that found or did not find "improved student scores." Unfortunately, narrative reviews fail to accumulate findings (Cooper, 1982), and do not allow the reader to compare different studies and outcome measures. Meta-analysis, on the other hand, provides standardized values for different outcome variables that can be quantified to a single numerical value known as an effect size-allowing the results of each study's dependent variables to be compared with other studies. …