Diagrams, drawings, and pictures are prototypical representations of concepts. Students' drawings of their concepts of convergent plate boundaries provided an efficient means or discovering some widely held misconceptions. Over 600 general education students' drawings of continent -continent convergent boundaries reveal two common misconceptions. Approximately one-third drew a continent-continent convergent boundary with concave slabs of continental crust as one might imagine two pieces of firm rubber pushed together on a rigid surface. Almost half drew mountains as one might imagine inverted ice cream cones on a rigid plank. One hundred eighty students were presented a drawing similar to the first misconception and asked to comment on what was incorrect. Forty-nine percent of the students did not recognize the misconception. Students who did not recognize the incorrect representation had lower scores on the Purdue Visualization of Rotations Test than the students who did. Misconceptions common to non-majors persist among upper class majors and beginning graduate students. Ten of 21 upper level geology majors and first year graduate students drew continent-continent convergent boundaries representing the same two prototype misconceptions that non-majors drew. Five out of 14 senior geology majors given a drawing representing a common misconception failed to recognize that the drawing was incorrect.
Evaluating the conceptual understanding of large numbers of individuals is difficult for lots of reasons. Limited degrees of understanding are difficult to distinguish from rote memorization and novices with prodigious memories can answer questions that might stump an expert. Expert understanding may be indicated by an ability to solve novel problems, but difficulty arises in determining degrees to which a problem or situation is novel for students. Also, understanding includes different categories of knowledge and cognitive skills. The phrase "cognitive skill" refers to tasks that require both declarative and procedural knowledge (Anderson, 1982). Declarative and procedural knowledge have characteristics that suggest that they function in different parts of the brain because amnesiacs may learn procedural knowledge but have no declarative memory of the learning process (Eichenbaum, 2002). It is common to find people who know how to get around town but can't provide directions. Tasks that require what is commonly referred to as higher order thinking involve both cognitive skills, which can be readily learned, and intellectual abilities or intelligence which is generally assumed to be less amenable to change through instruction (Royer et al., 1993). One means of distinguishing cognitive skill from intellectual ability is to think of cognitive skills as those skills which do not readily transfer to new domains and intelligence as those activities which do transfer. Of course, intelligence aides in developing cognitive skill, so the two are not clearly separable.
In view of the many kinds, levels and degrees of understanding, faculty must employ a variety of instruments that may probe students understanding. No single instrument is likely to effectively assess all aspects of understanding a concept. Most probes are analogous to uncalibrated analytical instruments (Haladyna, 1997). Interviews are the most informative means of probing individual students' understanding, but can only be clone with a very small number of students (Sadler, 1998) and, therefore, generalizations to a larger population must be supported by additional analysis. Objective tests can be a precise instrument for assessing conceptual understanding in large populations, but items should be preceded by open-ended questions or interviews to identify misconceptions that may be used as a basis for foils (Libarkin and Kurdziel, 200Ia). Even results of well-established multiple-choice tests such as the Force Concept Inventory (Hestenes et al. …