answers, and spreading the students out in the testing room reduced copying.
Another helpful precaution is vigilant monitoring during the test by an adequate number of proctors. Although Barnett and Dalton (1981) found that intelligent students cheated less in high-risk situations (e.g., when there were several proctors) and Baird (1980) found that women were much less likely to cheat in high-risk environments, students typically perceive proctoring as less effective than do teachers. Forty-eight percent of faculty but only 21% of students agreed that proctors watch carefully and consistently throughout tests (Barnett & Dalton, 1981).
This difference between instructors' and students' evaluations of the effectiveness of proctoring suggests that copying is difficult to detect. Although instructors think that they are vigilant, students believe that copying is undetectable, a result they attribute to inadequate supervision. Our own experience is that instructors who performed the following analysis were surprised by the amount of cheating suggested by the procedure. In addition, the analysis indicates that some students engaging in suspicious behavior, such as looking around frequently, are not cheating.
A final precaution that discourages copying is to assign seats that vary from one examination to the next and differ from students' lecture seats. This procedure ensures that students cannot plan to copy from friends or from students who are performing well in the course.
Using Error Similarity
Although these precautions are reasonable, instructors do not always have the resources to carry them out. To help discourage students from cheating on multiple-choice examinations, we developed a procedure that determines the similarity of errors for any pair of examinees. The procedure is implemented by a computer program that records the items both students got wrong and whether the number of same wrong answers was above a chance level. The analysis is restricted to errors because cheating is suggested if two students consistently choose identical wrong alternatives for the same items.
Cody (1985) described a similar procedure, and both Cody's method and the one described here are closely related to Angoffs (1974) Index B developed to detect cheaters. The advantage of our procedure is that it yields a critical value indicating whether cheating is likely.
To exemplify the procedure, assume that Student × made 25 errors and Student Y made 23 errors on a 60-item multiple-choice test with 5 alternative choices for each item. Also, assume that 20 of these errors involved the same items and that for 18 of these 20 items both students chose the same wrong alternative. Even though the answer sheets are not identical, an error-similarity analysis suggests that cheating is likely because the probability of choosing by chance 18 of the same wrong alternatives for 20 wrong items is on the order of 5 in a million. One must also use a seating chart to determine whether two students were sitting close enough to cheat and which student may be copying from the other.
The error-similarity analysis works like this: Once all of the answer sheets have been scored, the computer program counts the number of times all pairs of students chose the same incorrect alternative for all items. If all four incorrect alternatives of a five-alternative item were equally attractive, then the probability of two students choosing the same incorrect alternative for an item by chance would be 4/16 or 25. Because it is unrealistic to expect that all incorrect alternatives will be equally likely, the probability (P) of an identical error by two students on an item is computed by the program from the students' responses and typically has a value around .40.1 Using the binomial distribution (McNemar, 1962), when the probability that errors match on any item is P, then the probability for k of N item errors matching is:
Using the previous example of Student × and Student Y, assume that the probability (P) of any two students choosing the same wrong answer is .40. Then the probability of choosing 18 or more answers the same out of 20 wrong items is the probability of choosing 18, 19, or 20 answers the same by chance. Using the binomial distribution with P = .40 and N = 20, this probability can be represented as (20!/18! 2!) (.40)18 (.60)2 + (20!/19! l!) (.40)19 (.60)l + (20!/20! O!) (.40)20 (.60)0, which is equal to .000004700 + .000000330 + .000000011 = .000005031, or about 5 millionths.
Because computing binomial probabilities when N is large can be time consuming, a normal approximation to the binomial may be used if N P > 5 and N (1 − P) > 5 (McNemar, 1962). The value for the mean of this normal approximation is NP and the value of the standard deviation is. IN P (1 − P). For the binomial distribution from the just-cited example in which P = .40 and N = 20, the mean of the normal approximation is 8 and the standard deviation is 2.19. Because 18 identical answers were given for 20 wrong items, the standard normal value is z = (X − M)/SD = (17.5 − 8)/2.19 = 4.34. Using a standard normal table, the probability of obtaining 18 or more of the same wrong answers by chance is .000007 or about 7 millionths. This is close to the probability of 5 millionths computed previously using the binomial distribution.
The probability of choosing the same wrong answers by chance can be computed for every possible pair of students. If there are S students, then the number of comparisons is S (S − 1)/2. The probability computed for each pair of students indicates the probability with which their similar errors occur by chance. The error-similarity analysis is based on these probabilities. Although only probabilities need to be computed for the analysis, it is convenient to express the____________________