Assessing Higher Order Thinking in Mathematics

GERALD KULM

The nation's attention has recently been directed to the finding that many students have a great deal of difficulty in solving the simplest mathematics problems that require thinking beyond the retrieval of practiced algorithms ( Dossey et al., 1988; McKnight et al., 1987). The narrow focus on "back to basics" and the nearly unanimous decision by states and school districts in the late 1970's to settle for minimal competency in mathematics resulted in exactly the results that might have been expected. Students have learned how to do numerical computation at the expense of learning how to think and solve problems.

A byproduct of an era of competency- based education and accountability has been a focus on, if not an obsession with, testing. Falling SAT scores, poor showings in comparison with Japan and other developed countries, and failures on competency tests have resulted in even more tests at state and district levels. Tests are becoming the primary and widely accepted means for determining entry to, progress through, and exit from educational program of all types and levels. Satisfactory performance on tests is becoming the sole indicator of the "value added" by education.

A great deal of time, money, and effort is expended by commercial testing companies to produce standardized tests that satisfy all of the requirements for reliability, norms, and validity. But how valid are these tests in measuring what really does (or ought to) go on in the nation's mathematics classrooms? Earlier this decade, debate about tests was often centered on the alignment of tests with the curriculum. Studies showed that there was a poor match between specific topics emphasized in mathematics textbooks, standardized tests, and the curriculum ( Freeman et al., 1980). Results from state, national, and international tests provide another perspective on the mismatch between what students are taught and what they are tested on. A review of scores showed that students performed better on state assessments than on national and international tests in which the contents of the items may be unfamiliar ( Kulm, 1986b). The Second International Mathematics Study (IMS) found, for example, that some students had the opportunity to learn only as few as 40 percent of the items in some content categories ( McKnight et al., 1987).

But leaving the alignment problem aside, how well do standardized achievement tests define the nature of the mathematics that students should learn in our schools? The NCTM Curriculum and Evaluation Standards for School Mathematics has identified assessment standards, some of which seem difficult to attain with current approaches to standardized

-71-