Toward a Test Theory for Assessing Student Understanding
Robert J. Mislevy, Kentaro Yamamoto, and Steven Anacker
The view of learning that underlies standard test theory is inconsistent with the view rapidly emerging from cognitive and educational psychology. Learners become more competent not simply by learning more facts and skills, but by reconfiguring their knowledge; by "chunking" information to reduce memory loads; and by developing strategies and models that help them discern when and how facts and skills are important. Neither classical test theory (CTT) nor item response theory (IRT) is designed to inform educational decisions conceived from this perspective. This chapter sketches the outlines of a test theory built around models of student understanding, as inspired by the substance and the psychology of the domain of interest. The ideas are illustrated with a simple numerical example based on Siegler's balance beam tasks ( Siegler, 1981). Directions in which the approach must be developed to be broadly useful in educational practice are discussed.
When schooling became mandatory at the turn of the century, educators suddenly faced selection and placement decisions for unprecedented numbers of students, of diverse abilities and backgrounds ( Glaser, 1981). Numbers of correct answers to multiple-choice test items were used to rank students according to their overall proficiencies in domains of tasks. These rankings were used in turn to predict students' success in fixed educational experiences.