Redundant versus Consistent Stems in Multiple-Choice Vocabulary Tests and Their Effects on the Pre-University Students' Performance
Gorjian, Bahman, Jalilifar, Alireza, Mousavi, Halimeh, International Journal of Pedagogies and Learning
The goal of testing vocabulary is to assess the subjects' knowledge of lexical items. In preparing vocabulary tests, the first task concerns the selection of the vocabulary to be included in the test. In case of achievement testing, the test constructor chooses the vocabulary items from the materials covered in the course (Farhady, Jafarpur & Birjandi, 2004, p.179). Ballantyne (2004, p.1) argues that objective tests are made up of short questions, termed items. The most common test regarding vocabulary is the multiple-choice test. Since the construction of a stem in multiple-choice tests of vocabulary has a predominant function on the students' choice of correct alternative and gives a clear portrait of students' achievement during the course, this study will investigate using redundant stems and consistent stems in multiple-choice tests of vocabulary in order to see the effect of context provided in the stem. This may reveal the fact that contextualized items provide the test takers with more information about the situation, increase their cognitive extension, and also have beneficial backwash effect on their learning process.
Many of the researchers (e.g., Cooper, 2002; Brown, 1996) reject the use of redundancy items in the stems of multiple-choice items. There are also debates about the use of consistent stems, (i.e., simple sentence stems) in multiple-choice items. On the face of it, Heaton (1988, p.56) states that simple sentence stems do not provide enough contexts and too little contexts are insufficient to establish any meaningful situation. Anyway, there have been no comparative studies on the multiple choice vocabulary items with redundant or consistent stems. As a consequence, there is a gap in the experimental work on the relationship between these kinds of stems and their effectiveness which needs to be bridged. In order to draw up the boundaries of research, this study intended to pursue the following research questions:
1) Are multiple-choice vocabulary tests with redundant stems more effective than those of consistent stems?
2) Are multiple-choice vocabulary tests with consistent stems more effective than those with redundant stems?
The kind of context provided in the stem of a multiple-choice vocabulary item has a very important effect on the students' decision about choosing the correct alternative and also is a reliable criterion for the teachers to infer the students' knowledge of vocabulary.
Review of literature
A number of experimental studies (e.g., Muller, 2006) have been carried out on the stem orientation and stem formats as well as their effects on item difficulty and discrimination. In a research conducted by Violato and Marini (1989) the effects of stem orientation (positively stated stem or negatively stated stem) and completeness (complete stem or incomplete stem) of multiple-choice were investigated. Provisions were also made to determine possible interactions between orientation, completeness, and achievement. Thus, 142 senior education students were classified into three achievement groups (Low, Medium, High). A three-way factorial design (orientation. completeness. achievement) was used as the experimental model. Results indicated that incomplete versus complete stems increased item difficulty but had no effect on discrimination. Stem orientation had no effect on either difficulty or discrimination. Neither orientation nor completeness interacted with achievement or with each other.
In another research, Ascalon, Meyers, Davies, and Smits (2007) examined the format of the item stem and homogeneity of the answer set and their effects on item difficulty. A mock multiplechoice license examination was administered to high school students with items having item stems that were either open-ended or in question form and with distractors structured to be either similar or dissimilar to the correct answer. Analyses at the test level indicated that the similarity structured distractors raised the mean difficulty level by . …