Redundant versus Consistent Stems in Multiple-Choice Vocabulary Tests and Their Effects on the Pre-University Students' Performance
Gorjian, Bahman, Jalilifar, Alireza, Mousavi, Halimeh, International Journal of Pedagogies and Learning
This research attempted to determine the effect of redundant and consistent stems in multiple-choice vocabulary tests on pre-university students' performance. To this purpose, a sample English language proficiency tests was administered to a total of 130 pre-university students. Sixty homogeneous students were randomly selected and divided into two groups, one consistent group that took the test consisting of consistent stems and the other redundant group that took the test containing redundant stems. Next, three versions of the same test were provided in which the items had the same alternatives but different stems. The first and second versions of the consistent test were assigned to the consistent group, and the first two versions of the redundant tests to the redundant group. In the last versions of the two tests, the groups were reversed. This time, the consistent group took the test containing redundant stems while the redundant group took the test consisting of consistent stems. All the tests were teacher-made summative achievement tests and their reliability coefficients, the statistical independent t-test, paired samples t-test, and Pearson correlation formula were calculated. The results revealed that there was no significant difference between the students' performance on two kinds of tests (p<0.05). The empirical findings of the present study suggest that pre-university students' competence in vocabulary can be measured through both tests with redundant and tests with consistent stems.
Keywords: multiple-choice tests, redundant stems, consistent stems
The goal of testing vocabulary is to assess the subjects' knowledge of lexical items. In preparing vocabulary tests, the first task concerns the selection of the vocabulary to be included in the test. In case of achievement testing, the test constructor chooses the vocabulary items from the materials covered in the course (Farhady, Jafarpur & Birjandi, 2004, p.179). Ballantyne (2004, p.1) argues that objective tests are made up of short questions, termed items. The most common test regarding vocabulary is the multiple-choice test. Since the construction of a stem in multiple-choice tests of vocabulary has a predominant function on the students' choice of correct alternative and gives a clear portrait of students' achievement during the course, this study will investigate using redundant stems and consistent stems in multiple-choice tests of vocabulary in order to see the effect of context provided in the stem. This may reveal the fact that contextualized items provide the test takers with more information about the situation, increase their cognitive extension, and also have beneficial backwash effect on their learning process.
Many of the researchers (e.g., Cooper, 2002; Brown, 1996) reject the use of redundancy items in the stems of multiple-choice items. There are also debates about the use of consistent stems, (i.e., simple sentence stems) in multiple-choice items. On the face of it, Heaton (1988, p.56) states that simple sentence stems do not provide enough contexts and too little contexts are insufficient to establish any meaningful situation. Anyway, there have been no comparative studies on the multiple choice vocabulary items with redundant or consistent stems. As a consequence, there is a gap in the experimental work on the relationship between these kinds of stems and their effectiveness which needs to be bridged. In order to draw up the boundaries of research, this study intended to pursue the following research questions:
1) Are multiple-choice vocabulary tests with redundant stems more effective than those of consistent stems?
2) Are multiple-choice vocabulary tests with consistent stems more effective than those with redundant stems?
The kind of context provided in the stem of a multiple-choice vocabulary item has a very important effect on the students' decision about choosing the correct alternative and also is a reliable criterion for the teachers to infer the students' knowledge of vocabulary. …