Multiple-Choice Exam Question Order Influences on Student Performance, Completion Time, and Perceptions
Pettijohn, Terry F., II, Sacco, Matthew F., Journal of Instructional Psychology
We conducted 2 studies to investigate undergraduate performance, perceptions, and time required in completing sequentially ordered, randomly ordered, or reverse ordered exams in introductory psychology classes. Study 1 compared the outcomes and perceptions of students (N = 66) on 3 non-comprehensive multiple-choice exams which were sequentially, randomly, and reverse ordered and Study 2 investigated the outcomes and perceptions of students on a multiple-choice final exam. We also measured perceived test difficulty, test anxiety, and understanding of material. There were no statistically significant differences between the scores on the different exams or the time required to complete the exam versions, but perceptions of difficulty were influenced by the version of the exam assigned. Professors should consider these findings when testing students.
To prevent cheating on exams, many professors will mix up the order of multiple-choice test questions from exam to exam without thought of the consequence the order may have on student exam performance and perceptions. Textbook companies even provide randomization options for preparing exams using electronic test banks to assist in this common practice. Some research suggests that the different exam versions can have a significant effect on student performance. According to Balch (1989), students score higher on multiple-choice exams when the questions are presented in the same order that the material was presented in lecture and text as opposed to when questions are randomly grouped by chapter or in completely random order. Providing an advantage to one group of students who take the sequential versus a random test question order exam is problematic and unfair. Balch suggests that sequentially ordered exams provide retrieval cues which may help with memory recall, consistent with encoding specificity. The context of surrounding information used in encoding is utilized in information retrieval, and the sequential test question order provides a situation where context of encoding and retrieval are similar. In addition, Balch found that there was no significant difference in completion times between these versions of the exam. Other researchers have challenged this rationale and these findings.
Neely, Springston, and McCann (1994) conducted a three study follow-up to Balch (1989) in which student performance on sequential and random order multiple-choice question exams in an introductory psychology class were compared and the influence of test anxiety was also considered. The results of the three studies showed no significant difference between the sequential and random order multiple-choice question tests. However, the researchers did report a significant interaction such that high-anxiety students performed "somewhat better" on the sequential question order test and low-anxiety students performed "substantially better" on the random question order test. Similarly, Peters and Messier (1970) also found no differences in performance on sequential versus random question order multiple-choice tests in a class of graduate students studying research methods, and those students who reported high levels of anxiety performed worse on the random question order test compared to the sequential question order test.
Perlini, Lind, and Zumbo (1998) further investigated the effects of item order and item difficulty on test performance in four studies. In experiment 2, the investigators found no advantage in student performance on sequenced chapter-order multiple-choice question tests over random or reverse question order tests. In experiment 3, researchers varied chapter question order and within chapter question order, but again found no performance differences between conditions. Item and chapter question arrangements were found to have little or no effect on test performance. Perlini, Lind, and Zumbo also arranged test questions with respect to item difficulty in their fourth study: easy-to-hard, hard-to-easy, or random. …