Academic journal article Kuram ve Uygulamada Egitim Bilimleri

Investigating Item Exposure Control Methods in Computerized Adaptive Testing*

Academic journal article Kuram ve Uygulamada Egitim Bilimleri

Investigating Item Exposure Control Methods in Computerized Adaptive Testing*

Article excerpt

(ProQuest: ... denotes formulae omitted.)

The administering of Computerized Adaptive Testing (CAT) has increased, and continue to do so, in line with concurrent developments in both the fields of computer technology and psychometrics. In lieu of traditional paper-and-pencil tests, CAT style tests have become increasingly attractive because they are easy to apply and to score, there are items in the tests corresponding to examinees' ability levels, such tests can be administered whenever desired, and the tests are shorter when compared to traditional paper-and-pencil tests (Grist, Rudner & Wise, 1989; Meijer & Nering, 1999; Rudner, 1998; Weiss & Kingsbury, 1984). CAT applications are used in the administration of the GMAT (Graduate Management Admission Test) and the GRE (Graduate Record Examination). As in traditional paper-and-pencil tests, these applications, also need to be reliable and valid since they are used to make critical decisions regarding the futures of examinees. As the use of CAT have become increasingly widespread, so have a number of problems with the potential of endangering validity appeared. Accordingly, the issue of such tests' security increases in importance (Chang & Twu, 1998; Davey & Nering, 2002 as cited in Barrada, Olea, Ponsoda, & Abad, 2009; French & Thompson, 2003; Georgiadou, Triantafillou, & Economide, 2007; Lee & Dodd, 2012).

Although CAT applications require item pools containing a large number of items, certain items are used more frequently than others in specific situations. In such situations, the probability of examinees' attempting simply to memorize the answers to frequently used items is higher. If such items are memorized and then shared, the test's validity becomes jeopardized (Georgiadou et al., 2007; Lee & Dodd, 2012).

Since developing an ideal item pool is the result of a long and laborious process, it is undesirable for test developers to use just a certain percentage of the item pool, instead it would be better to use the entire pool efficiently (Revuelta & Ponsoda, 1998). For the stated reasons, a number of methods were developed so as to assure test security and to make use of the item pool more efficiently. Such methods are called "item exposure control methods," which have gradually been included in the fundamental components of CAT due to the problems encountered in real-life applications (Boyd, 2003; Davis, 2002).

Newly developed item exposure control methods were added to the existing ones so as to prevent such issues as examinees' being repeatedly exposed to the same items, the same items being administered to examinees of similar proficiency levels, a low percentage of the item pool being used, and the inclusion of frequently used items of the pool in diverse applications after being revealed.

The results of the newly developed methods should be compared with existing ones. The results to be yielded by the item exposure control methods (selection method, the psychometric properties of the items, the size of the item pool, the termination rule, and sampling distribution) in changeable factors during the administration of CAT remain an issue requiring research.

The most important aspect in CAT and in item exposure control is the item pool because whether a successful test may be conducted or not depends on the properties of the items within the pool. With this in mind, one of the factors that influences item exposure is the psychometric properties of the items in the item pool (Revuelta & Ponsoda, 1998). From a psychometric perspective, however, items are assessed based on the item information corresponding to their item parameters and to the ability levels. The desired distribution of the items along the ability scale varies according to the goal of the test. While the item pool for achievement tests should contain a variety of items ranging from "very easy" to "very difficult," the ideal distribution of the b parameter should be uniform distribution. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.