Academic journal article Psychological Test and Assessment Modeling

Using Rasch Model Generalizations for Taking Testee's Speed, in Addition to Their Power, into Account

Academic journal article Psychological Test and Assessment Modeling

Using Rasch Model Generalizations for Taking Testee's Speed, in Addition to Their Power, into Account

Article excerpt

Introduction

In psychological assessment, the speed-power-issue has existed almost since the beginnings of psychological testing. Most intelligence tests have a time-limit, which is often only due to organizational reasons (to make test administration possible for a group of testees, for instance). Apart from this, some intelligence and achievement tests involve speed by scoring. For example, the commonly used Wechsler tests (e.g. Wechsler Adult Intelligence Scale - Fourth Edition, WAIS IV; Wechsler, 2008) include subtests that credit quick solutions with bonus points. The desirable advantage of such a scoring procedure is an attainment of information about a testee's ability. The scores of the testees are more differentiated and thus, measurement would take place in a more precise way. Of course, the advantage of such information is only valid if the assumptions underlying the scoring procedure are correct. Particularly, using bonus points assumes that "power" and "speed" are confounded but not separated traits. This means the ability to solve an item and speededness of a testee in finding the solution are assumed to be a manifestation of the same latent trait and reflect only gradual differences in the measured trait. This assumption is to be scrutinized, as there are empirical results that show speed and power are actually separated (Carroll, 1993; Partchev, De Boeck, & Steyer, 2011). Partchev, De Boeck, and Steyer (2011) even remark that the current focus should be on avoiding a mixture between speed and power. However, in practice, scoring procedures combining speed and power do still exist, which makes the application of methods which test the validness of such a scoring procedure important.

Nowadays, it is easy to record response times for each item and there are increasing attempts to use these response times not only in psychological but also in educational assessment. Large-scale tests that have been applied for years, are currently going through a transition from paper-based administration to a computer-based one. In the first part of the special topic "Current Methodological Issues in Educational Large-Scale Assessments" by Stadler, Greiff, and Krolak-Schwerdt (2016) in this journal Bürger, Krohne, and Goldhammer (2016) give a short overview which of the broad-based international large-scale assessments have already changed their administration mode or are planning to change it in the near future. This transition to a computer-based administration provides the opportunity to record not only the response of the examinee but many other variables including the item-specific response time. Response times provide an additional information about how the testee did work on the test. So far, there are many attempts to use response times to increase the measurement accuracy and to minimize measurement errors in psychological and educational assessment. For example, a variety of studies dealt with the detection of guessing in multiple-choice items by using the response times (DeMars, 2007, 2010; Kong, Wise, & Bhola, 2007; Schnipke, & Scrams, 2007, Wise, Pastor, & Kong, 2009). Weeks, von Davier, and Yamamoto (2016) are using response times to distinguish between missing responses which were skipped and those the testee had tried on which but did not give a response.

Another option to use the information of response times in large-scale assessment is to incorporate the response times into scoring; that is analogous to intelligence tests which use some credit points for quick solutions. This approach is thought of as a means of increasing measurement accuracy which is of need especially in large-scale assessments where the number of items is limited due to organizational restrictions.

A variety of approaches were introduced for incorporating response times in assessments. Van der Linden (2011) gives a fine overview of actual IRT methods modeling response times. He distinguishes between models that include the distributions of response times without any reference to the quality of the item response, and models that integrate item responses and response times (e. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.