Academic journal article Psychological Test and Assessment Modeling

Sequentially Presented Response Options Prevent the Use of Testwiseness Cues in Multiple-Choice Testing

Academic journal article Psychological Test and Assessment Modeling

Sequentially Presented Response Options Prevent the Use of Testwiseness Cues in Multiple-Choice Testing

Article excerpt


Multiple-choice testing is one of the most popular testing formats for the assessment of knowledge. It is widely used in diverse settings including school tests, university exams, vocational aptitude tests, and even TV quiz shows. In its standard form, a multiple-choice (henceforth MC) item consists of a stem and a set of three to five response options, one of which is the solution (Foster & Miller, 2009). The stem is the core of an item, which presents the question that has to be answered. Next to the stem, all possible response options are presented. The examinee's task is to choose the correct answer from among this set of options. Sometimes this variant of MC testing is called "single-choice" testing because only one response option is the correct solution. Usually, all options (i.e., the solution and the distractors) are presented simultaneously to the test taker.

MC testing of this kind provides an efficient way to objectively measure cognitive ability. Unlike other test formats such as open questions or essays, MC tests can be scored easily, objectively, and even in an automated manner, rendering the testing of large groups feasible (Tamir, 1991). Considering the approximately 90 years of research on MC tests, Downing (2006) concluded that there is strong evidence for the validity of MC testing across a wide range of areas.

Critics, however, have doubted that recording the mere selection of a MC response option adequately assesses higher order thinking skills (Hancock, 1994). The selection of an MC option may not reveal actual knowledge of a respondent, but simply indicate the alternative a respondent considers to be the most plausible (Holmes, 2002). This choice is based on a comparison that is performed by taking all available options into account simultaneously. Therefore, a drawback of the MC test format is that cues that indicate which solution is correct may be derived or identified by comparing the various response options.

Gibb (1964) defined testwiseness as the ability to find and to make use of such extraneous cues in MC items. Item cues have been shown to make MC items less difficult, and testwise persons who are capable of making use of item cues may use these cues to increase their test scores (Allan, 1992). Rost and Sparfeldt (2007) surprisingly found that by comparing all available response options, pupils could often identify the correct solution without even knowing the question (cf. also Sparfeldt, Kimmel, Löwenkamp, Steingräber & Rost, 2012).

Item cues that can be used to identify the correct answer also reduce the construct validity of MC items if individual differences in testwiseness - that need not necessarily be related to the examinee's knowledge - add construct-irrelevant variance to MC test scores (Haladyna & Downing, 2004; Millman, Bishop & Ebel, 1965; Rost & Sparfeldt, 2007). In principle, items on carefully constructed tests should not be solvable by simply using testwiseness strategies if guidelines for good item writing practices are followed (Haladyna, 2004). However, many MC items are created under time pressure and by authors who have little experience with test development (Downing, 2006). Accordingly, Brozo, Schmelzer, and Spires (1984) found that even in a sample of 1,220 MC items that had been used in real college examinations, 44 % of the items contained one of 10 different kinds of item cues. On average, for these flawed items, using the available cues almost tripled the probability of a correct solution as compared to a baseline of random guessing. Several other investigations also showed a high prevalence of item flaws that allowed identifying the solution (e.g. Hughes et al., 1991; Metfessel & Sax, 1958; Tomkowicz & Rogers, 2005). In a more recent study, Tarrant and Ware (2008) analyzed 10 tests that had been used for high-stakes assessments in a nursing program. They also found that between 28 - 75 % of the MC test items contained flaws, most of which favored testwise students. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.