The research question was: When a non-paper-and-pencil test is used as the source of measurement, do there appear to be differences by students' mathematics curriculum, Algebra II or CPMP III, in the mastery level of problem-solving and algebra? Students from 3 classes of CPMP III and 3 classes of Algebra II (resulting in 76 CPMP III students and 71 Algebra II students), all in the same school district, were given a four-item interview test. To answer the research question only in terms of correctness, there was equal performance on two non-routine problems. On two algebraic items, the Algebra II students scored significantly higher. In terms of Polya's stages entered (did not attempt, understanding the problem, devising a plan, carrying out the plan, looking back), there were no significant differences between the two groups, nor was there a difference in terms of strategies employed. CPMP III students took significantly more time in completing the test than the Algebra II students did.
CPMP III versus Algebra II
Standards-oriented mathematics curricula are present in numerous secondary mathematics classrooms (National Council of Teachers of Mathematics, 2000). These curriculum projects were based on research and project personnel continue to do extensive research on student outcomes (Senk & Thompson, 2003). It may appear then that further research on standards-oriented curricula is not necessary. However, the vast majority of the research has been conducted by project personnel. Although no one is accusing project personnel of faulty or unethical research, it is a concern that researcher bias may occur to some extent when high stake studies are conducted by the very people with high stakes (Campbell & Russo, 1999; Crano & Brewer, 2002; Mehra, 2002).
Berg (1988) cautions that "[i]t is misleading to assume that because researchers are scientists they are not human beings. Research is an activity embedded in a profession.... As a result, all of the anxieties associated with earning a living (e.g., advancement, security, recognition) are also part of the research process." (p. 219). Qualitative research has taught us that "all research is participatory" (Brown, 1996, p. 15) and that as researchers conduct research they "naturally experience a range of emotions and thoughts, some of which threaten to bias, distract, and even disable them" (Brown, 1996, p. 15).
Researcher bias in studies of standards-oriented curricula versus traditional curricula can occur in several manners, and may even seem appropriate to the reader. For example, it is common to have researchers select teachers who most faithfully implement the standards-oriented curricula. At first glance, this might seem appropriate. However, teachers are implementing standards-oriented curricula across the nation. If it takes a perfectly committed and extensively trained teacher to implement the curriculum successfully, then it should be considered whether the curriculum is a realistic curriculum to use. It would be better to attempt to select teachers in a random fashion. If teachers are selected in a random fashion, the studies will not be so much about what is possible, as they will be about what is happening. Thus it is timely that more non-project researchers conduct studies regarding the student outcomes in standards-oriented curriculum. This paper documents one such study.
Besides having this study conducted by a researcher not connected to a standards-oriented project, this study contributes to the research in another significant manner. Some of the existing research reports on qualitative results using a single standards-oriented curriculum (without comparison to another standards-oriented curriculum, or to a traditional curriculum). Other studies offer comparisons but compare quantitative results of standards-oriented students' scores on paper-and-pencil tests to traditional students' scores (Kilpatrick, 2003). …