Academic journal article Journal of Education and Learning

A Proposal on the Validation Model of Equivalence between PBLT and CBLT

Academic journal article Journal of Education and Learning

A Proposal on the Validation Model of Equivalence between PBLT and CBLT

Article excerpt

Abstract

The validity of the computer-based language test is possibly affected by three factors: computer familiarity, audio-visual cognitive competence, and other discrepancies in construct. Therefore, validating the equivalence between the paper-and-pencil language test and the computer-based language test is a key step in the procedure of designing a computer-based language test. By taking the test on Essentials of English-Speaking Countries as the case study, this paper elucidates the three-step model of validating the equivalence of the two types of test: investigating computer familiarity, assessing the impact of audio-visual cognitive competence, and examining other discrepancies in construct. The model proposed by this paper can offer some methodological insights on the way to establishing the validation model of the equivalence between the paper-and-pencil language test and the computer-based language test.

Keywords: paper-and-pencil language test, computer-based language test, test equivalence, validation model

1. Introduction

According to Bachman, "Validation has been, and continues to be, a recurring theme of the annual Language Testing Research Colloquium ... validation has become the de facto paradigm for language testing research and development".(2000, p. 22) Since the 1980s, the newly developed Computer Based Language Testing (CBLT) has been gradually replacing the conventional Paper-and-Pencil Based Language Testing (PBLT). A key problem encountered in computerized testing is whether an item bank designed for the conventional PBLT can be adopted as the item bank for CBLT with the test validity basically unchanged.

To solve that problem, a lot of studies were conducted to investigate the possibility of validating the equivalence between PBLT and CBLT. Henning (1991) elucidated the validation challenges encountered in constructing the item bank for CBLT by focusing on test methods and procedures. Wainer (2000), however, only focused on method effects. Moreover, Li (2006) held that research on equivalence between PBLT and CBLT should "explore the equivalence or comparability of the results of the same test tasks under different modes of test presentation (in print or on screen) and different response modes (paper and pencil or mouse and keyboard)". The previous studies mentioned above have different research focuses probably because they might have adopted different definitions of test validity. The prevailing definition of test validity was put forward by American Psychological Association which defined test validity as "the appropriateness, meaningfulness, and usefulness of the specific inferences made from test scores" (1999, p. 9). According to that authoritative definition, this study holds that the research on validating the equivalence between PBLT and CBLT should take both test construct and method effects into consideration and we should not only see the challenges but also formulate a practical procedure to validate the equivalence between PBLT and CBLT.

2. Framework and Procedure

Based on the aforementioned studies, this study holds that the difference in validity between PBLT and CBLT may be brought about by computer familiarity, audio-visual cognitive competence, and test construct. Therefore, in this study we consider a three-step procedure to validate the equivalence between PBLT and CBLT. Step one aims to estimate the impact of examinees' computer competence on their test results with reference to their levels of computer familiarity. Step two aims to estimate the impact of examinees' audio-visual competence by comparing the test results between PBLT and multimedia CBLT. Step three aims to detect other construct discrepancies by comparing the construct validity of PBLT and CBLT. The three steps are sequentially interlocked and thus should be conducted in a sequential order.

2.1 Impact of Computer Familiarity

An important aspect which needs to be taken into consideration for validating CBLT is the computer familiarity of the examinees. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.