Academic journal article Phi Delta Kappan

Educators Should Require Evidence

Academic journal article Phi Delta Kappan

Educators Should Require Evidence

Article excerpt

In renewing schools, developing staff, and advocating literacy, Bruce Joyce may not have had time to find and read the negative evaluations of Success for All, Mr. Walberg and Ms. Greenberg respond. If so, perhaps his statements should be more cautious.

Delighted we are to see the Kappan reprint our Education Week commentary and to respond to Bruce Joyce's self-declared perturbation. We wrote partly in response to Olatokunbo Fashola and Robert Slavin's January 1998 Kappan article, which concluded that Success for All (SFA) yields better results than its competitors.1 Program designers who evaluate their own programs, however, may have conflicting interests.

Our purpose here is to answer Mr. Joyce about research evidence and its role in choosing programs, not to describe the reasons why teachers have found Success for All ineffective and objectionable. For such comments, we refer readers to the Internet site, especially the section titled "Broken Promises."

SFA developers' omission of their own and others' negative evidence about SFA furthered our suspicions. We hardly suspected dishonesty but perhaps unconscious bias, long recognized in the sciences. As we pointed out, medical researchers employ double-blind experiments to eliminate such bias in detecting the real effect of unproven treatments. They look for a pure effect independent of the investigator's, nurse's, or doctor's preconceptions and other psychological conditions that may affect patients' recovery. Such bias has long been recognized in educational research.2 More generally, bias seems to be part of human nature. Chicagoans warn, "Don't ask your barber whether you need a haircut." Skepticism (not necessarily cynicism) goes back to classical times: when considering what action to take, Aristotle cautioned his students to consider the source of information, and the Romans asked, "Cui bono?" (Who benefits?)

Contrary to Joyce's interpretation of our view, however, we have no problem with monetary motives; we celebrate the market economy. But, as the Romans warned, "Caveat emptor." Who takes for granted a General Motors claim that its cars are better than Fords or Toyotas? Consumer Reports or a test drive might provide useful independent information.

What's the Evidence? Similarly, educators might seek independent evidence of programs' success. They might even conduct their own research on programs they've adopted with great expectations. The Miami-Dade County schools, for example, spent $50 million of Title I funds on SFA and continued to spend an annual $50,000 per school. If SFA did better than other programs and brought third-graders in poverty to national norms as claimed, it would be money well spent. Six months or so after our commentary appeared, however, Miami-Dade's independent evaluation showed that SFA did no better than other programs. In fact, for second-graders, phonics-based direct instruction did better than SFA.3

Notwithstanding such negative evidence or lack of evidence, we rarely meet program designers who say that other programs are better than their own. Similarly, it may not be surprising that, as a self-declared staff developer, Joyce sees the need for more staff development. Self-advocates, of course, may be right. They may also be motivated by zealotry, money, habit, truth, altruism, mistaken beliefs, the desire to lead others, or perhaps a combination of factors.

Though he offers no evidence, Joyce says SFA works. He also cites two reading programs (apparently his own), which, he says, have elements in common with SFA. One of these, he says, "has had considerable success with children of all ages." Perhaps he may be available to develop staff to implement these programs around the country.

Joyce says he also specializes in school renewal. But in addition to his self-testimonials, we'd like to see control-group research in support of his claims about the efficacy of particular programs. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.