with an effective routine. It is no wonder that this conception isolates mathematics from other subjects, since what is here described is not so much a form of thinking as a substitute for thinking. The process of calculation or computation only involves the deployment of a set routine with no room for ingenuity or flair, no place for guesswork or surprise, no chance for discovery, no need for the human being in fact. (p. 184)
The single, most severe criticism of objective test questions that are designed to assess a specific item of content at a specific level of behavior is that they trivialize learning and knowledge ( Berlak, 1985). For several reasons, this is almost inherent in such questions. First, they are constructed to test a single, specific objective, clearly defined in the matrix. Thus, elements in the multiple-choice format are designed so that the candidate can pick an answer which is sufficiently specific to unequivocally demonstrate the sought behavior. This tends to eliminate any synthesis between content and behavior. Second, the very nature of objective tests, which ask the user to choose among alternatives, eliminates creativity in answering. Even the intent militates against creativity because it is microanalytical rather than synthetic or creative.
Frederiksen ( 1984) observed that a multiple-choice format does not measure the same cognitive skills as a free-response form and that "efficient tests tend to drive out less efficient tests, leaving many important abilities untested -- and untaught" (p. 201). One example of a desirable outcome untested and untaught is the ability to cope with ill-structured problems, which are not found on standardized achievement tests.
A less obvious impact was observed by the Assessment of Performance Unit ( Cambridge Institute of Education, 1985). The multiple- choice format is an interventional mode of questioning which appears to offer a greater chance for success in situations where the student is unfamiliar with the material. However, in other situations, students benefitted from the opportunity to think, achieving greater success with the free-form response.
Another aspect of most objective tests is that, even though some questions may be designed to test lower level thinking and others are designed to evaluate higher thought processes, they are usually tested independently of each other, allowing little notion of a student's approach to a given problem.
In addition to their direct effects, such tests exert powerful indirect effects on both the style of teaching and the style of learning. When one studies for an essay exam, one progressively surveys and synthesizes, putting the parts together and developing a mental model of the structure of the subject. One also develops points of view and arguments to advance and support, for those are the expectations. By contrast, in studying for an objective, multiple-choice test, one learns to cover the parts and to make fine distinctions between alternative ways of stating the same thing in order to distinguish a "right" answer from a "wrong" one, the implication being that there is a single right answer. In other words, the one form requires that students create their own models of mathematics, the other reinforces the view of mathematics as ground to be covered.
Unfortunately, it is incredibly difficult to shrug off old habits. For example, the architecture of current evaluations -- the two- dimensional, content-by-behavior matrix -- is a seductively convenient model for organizing information visually and for reducing the inherent complexity of relationships, which we know give a false sense of simplicity. Occasionauy, a three-dimensional version expands the possibilities (e.g., Carpenter, Coburn, Reys, & Wilson, 1978; Foxman, Cresswell, & Badger, 1981), but increases the conceptual load and so is used less. The intellectual consequences of using a two-dimensional matrix deserve thought. Such a matrix encourages a tendency to tacitly view successive cells in a row or column as entities having a sequential and linear relationship to each other. It also causes visual separation of nonadjacent cells, subliminally interrupting perception of relationships between them. If such relationships do exist, the visual patterns of the matrix have a powerful often mnemonic, impact. If