Handbook for Teaching Introductory Psychology: With An Emphasis on Assessment - Vol. 3

By Richard A. Griggs | Go to book overview
Save to active project

answers, and spreading the students out in the testing room reduced copying.

Another helpful precaution is vigilant monitoring during the test by an adequate number of proctors. Although Barnett and Dalton (1981) found that intelligent students cheated less in high-risk situations (e.g., when there were several proctors) and Baird (1980) found that women were much less likely to cheat in high-risk environments, students typically perceive proctoring as less effective than do teachers. Forty-eight percent of faculty but only 21% of students agreed that proctors watch carefully and consistently throughout tests (Barnett & Dalton, 1981).

This difference between instructors' and students' evaluations of the effectiveness of proctoring suggests that copying is difficult to detect. Although instructors think that they are vigilant, students believe that copying is undetectable, a result they attribute to inadequate supervision. Our own experience is that instructors who performed the following analysis were surprised by the amount of cheating suggested by the procedure. In addition, the analysis indicates that some students engaging in suspicious behavior, such as looking around frequently, are not cheating.

A final precaution that discourages copying is to assign seats that vary from one examination to the next and differ from students' lecture seats. This procedure ensures that students cannot plan to copy from friends or from students who are performing well in the course.


Analysis of Multiple-choice Items
Using Error Similarity

Although these precautions are reasonable, instructors do not always have the resources to carry them out. To help discourage students from cheating on multiple-choice examinations, we developed a procedure that determines the similarity of errors for any pair of examinees. The procedure is implemented by a computer program that records the items both students got wrong and whether the number of same wrong answers was above a chance level. The analysis is restricted to errors because cheating is suggested if two students consistently choose identical wrong alternatives for the same items.

Cody (1985) described a similar procedure, and both Cody's method and the one described here are closely related to Angoffs (1974) Index B developed to detect cheaters. The advantage of our procedure is that it yields a critical value indicating whether cheating is likely.

To exemplify the procedure, assume that Student × made 25 errors and Student Y made 23 errors on a 60-item multiple-choice test with 5 alternative choices for each item. Also, assume that 20 of these errors involved the same items and that for 18 of these 20 items both students chose the same wrong alternative. Even though the answer sheets are not identical, an error-similarity analysis suggests that cheating is likely because the probability of choosing by chance 18 of the same wrong alternatives for 20 wrong items is on the order of 5 in a million. One must also use a seating chart to determine whether two students were sitting close enough to cheat and which student may be copying from the other.


Probability of Same Errors Using the
Binomial Distribution

The error-similarity analysis works like this: Once all of the answer sheets have been scored, the computer program counts the number of times all pairs of students chose the same incorrect alternative for all items. If all four incorrect alternatives of a five-alternative item were equally attractive, then the probability of two students choosing the same incorrect alternative for an item by chance would be 4/16 or 25. Because it is unrealistic to expect that all incorrect alternatives will be equally likely, the probability (P) of an identical error by two students on an item is computed by the program from the students' responses and typically has a value around .40.1 Using the binomial distribution (McNemar, 1962), when the probability that errors match on any item is P, then the probability for k of N item errors matching is:

Using the previous example of Student × and Student Y, assume that the probability (P) of any two students choosing the same wrong answer is .40. Then the probability of choosing 18 or more answers the same out of 20 wrong items is the probability of choosing 18, 19, or 20 answers the same by chance. Using the binomial distribution with P = .40 and N = 20, this probability can be represented as (20!/18! 2!) (.40)18 (.60)2 + (20!/19! l!) (.40)19 (.60)l + (20!/20! O!) (.40)20 (.60)0, which is equal to .000004700 + .000000330 + .000000011 = .000005031, or about 5 millionths.

Because computing binomial probabilities when N is large can be time consuming, a normal approximation to the binomial may be used if N P > 5 and N (1 P) > 5 (McNemar, 1962). The value for the mean of this normal approximation is NP and the value of the standard deviation is. IN P (1 P). For the binomial distribution from the just-cited example in which P = .40 and N = 20, the mean of the normal approximation is 8 and the standard deviation is 2.19. Because 18 identical answers were given for 20 wrong items, the standard normal value is z = (X M)/SD = (17.5 8)/2.19 = 4.34. Using a standard normal table, the probability of obtaining 18 or more of the same wrong answers by chance is .000007 or about 7 millionths. This is close to the probability of 5 millionths computed previously using the binomial distribution.

The probability of choosing the same wrong answers by chance can be computed for every possible pair of students. If there are S students, then the number of comparisons is S (S 1)/2. The probability computed for each pair of students indicates the probability with which their similar errors occur by chance. The error-similarity analysis is based on these probabilities. Although only probabilities need to be computed for the analysis, it is convenient to express the

____________________
1
If there are only four alternative answers for each item, as is true of many items provided with introductory psychology texts, then P may be higher than .40. The program automatically takes the number of alternatives into account, and the procedure remains the same.

-161-

Notes for this page

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
Loading One moment ...
Project items
Notes
Cite this page

Cited page

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited page

Bookmark this page
Handbook for Teaching Introductory Psychology: With An Emphasis on Assessment - Vol. 3
Table of contents

Table of contents

Settings

Settings

Typeface
Text size Smaller Larger
Search within

Search within this book

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

While we understand printed pages are helpful to our users, this limitation is necessary to help protect our publishers' copyrighted material and prevent its unlawful distribution. We are sorry for any inconvenience.
Full screen
/ 257

matching results for page

Cited passage

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited passage

Welcome to the new Questia Reader

The Questia Reader has been updated to provide you with an even better online reading experience.  It is now 100% Responsive, which means you can read our books and articles on any sized device you wish.  All of your favorite tools like notes, highlights, and citations are still here, but the way you select text has been updated to be easier to use, especially on touchscreen devices.  Here's how:

1. Click or tap the first word you want to select.
2. Click or tap the last word you want to select.

OK, got it!

Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.

For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

Already a member? Log in now.

Are you sure you want to delete this highlight?