Academic journal article Psychological Test and Assessment Modeling

The Sequential Probability Ratio Test for Multidimensional Adaptive Testing with Between-Item Multidimensionality

Academic journal article Psychological Test and Assessment Modeling

The Sequential Probability Ratio Test for Multidimensional Adaptive Testing with Between-Item Multidimensionality

Article excerpt

Abstract

It is examined whether the unidimensional Sequential Probability Ratio Test (SPRT) can be productively combined with multidimensional adaptive testing (MAT). With a simulation study, it is investigated whether this combination results in more accurate simultaneous classifications on two or three dimensions compared to several instances of unidimensional adaptive testing (UCAT) in combination with SPRT. The number of cut scores, and the correlation between the dimensions measured were varied. The average test length was mainly influenced by the number of cut scores (one, four) and the adaptive algorithm (MAT, UCAT). With MAT, a lower average test length was achieved in comparison to the UCAT. It is concluded that MAT will result in a higher percentage of correct classifications than UCAT when more than two dimensions are measured.

Key words: classification, computerized adaptive testing, item response theory, multidimensional adaptive testing, sequential probability ratio test

(ProQuest: ... denotes formulae omitted.)

Multidimensional adaptive testing (MAT) is a special approach to the assessment of two or more latent abilities in which the selection of the test items presented to the examinee is based on the responses given by the examinee to previously administered items (e.g., Frey & Seitz, 2009). The main advantage of MAT is its capacity to substantially increase measurement efficiency compared to sequential testing or unidimensional computerized adaptive testing (UCAT). Most of the studies on MAT are focusing its application for assessing individual abilities located on continuous scales. Currently, only very little is known about the capabilities of MAT regarding the classification of test takers to one of several ability categories (e.g., pass vs. fail). To fill in this gap, the present paper focuses on the combination of MAT with the sequential probability ratio test (SPRT; e.g., Kings-bury & Weiss, 1983; Reckase, 1983). The SPRT is a classification method that already has been used successfully in combination with UCAT (e.g., Eggen, 1999; Eggen & Straetmans, 2000; Spray & Reckase, 1996; Thompson, 2007b).

Regarding MAT, Spray, Abdel-fattah, Huang, and Lau (1997) made an attempt to modify the SPRT in order to use it with MAT based on items with within-item multidimensionality. Items with within-item multidimensionality are allowed to measure more than one dimension simultaneously (Wang, Wilson, & Adams, 1997). Dealing with within-item multidimensionality, the multidimensional item response theory (IRT) model used with MAT is a compensatory model (e.g., Reckase, 2009). With such an IRT-model, the linear combination of the abilities measured leads to a curvilinear function. Therefore, the test statistic of the SPRT, which is a likelihood ratio test, cannot be updated by two unique values required by the SPRT. For details, see Spray et al. (1997). Considering multidimensional pass-fail tests, Spray and colleagues did not find a satisfactory solution for implementing a multidimensional SPRT into such a MAT.

Nevertheless, from a practical point of view, tests entailing items measuring exactly one dimension each (between-item multidimensionality) are much more common than tests based on an item pool with within-item multidimensionality. Hence, the present paper focusses on the combination of MAT and SPRT for items with between-item multidi-mensionality. Note that when the MAT approach of Segall ( 1996) is used for items with between-item multidimensionality, information from items which measure one dimension is used as information about the person's score on other dimensions. This is done by incorporating assumption about the multivariate ability distribution in terms of correlations between the measured dimensions. Several studies showed that using this information results in substantial increase in measurement efficiency compared to using several unidimensional adaptive tests (e. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.