Academic journal article Foreign Language Annals

Exploring Proficiency-Based vs. Performance-Based Items with Elicited Imitation Assessment

Academic journal article Foreign Language Annals

Exploring Proficiency-Based vs. Performance-Based Items with Elicited Imitation Assessment

Article excerpt

(ProQuest: ... denotes non-US-ASCII text omitted.)

The first recorded standardized tests were designed to solve a problem that occurred in imperial China. That problem? Unqualified job candidates were being appointed based on cronyism instead of merit. An unknown functionary hypothesized that a small sample of behavior gathered under carefully controlled circumstances would predict future behavior in uncontrolled situations (Wainer, Bradlow, & Wang, 2007). This observation led to the first standardized job screening tests that were used to select candidates based on external standards by evaluating a desired set of skills. Getting a job became a product of what one knew instead of whom one knew, and thus the first standards-based testing was born.

The overarching issue of testing since then has been determining what the "small sample of behavior" should look like and how it should be operationalized. Oral Proficiency Interviews (OPIs) have long been considered one of the most valid and reliable measures of standards-based (i.e., criterionreferenced) speaking ability, yet they are expensive and time consuming to administer, making them impractical in many instances. In order to find an alternative to the OPI, scholars have begun investigating the validity and utility of elicited imitation (EI) testing instruments. EI approaches language assessment through a series of itemrepetition tasks that intend to predict a testtaker'slanguageproficiency. Test-takers listen to a recording of a native speaker of the target language reading the item prompt, then are expected to immediately repeat what they heard as accurately as they are able (Chaudron, Prior, & Kozok, 2005). While on the surface this may seem like a memory exercise, many studies have provided convincing evidence that EI tests actually measure test-takers' interlanguage system (Bley-Vroman & Chaudron, 1994; Erlam, 2006; Perkins, Brutten, & Angelis, 1986). For a more complete discussion regarding what EI actually measures, consult Cox and Davies (2012). The behavioral what that is being measured by EI, thus, is artificial, in the sense that rarely is one required to repeat verbatim what one hears. However, individuals who have greater proficiency in the language are better able to repeat complex items (Burdis, 2014), and correlations between OPIs and EIs are reported later in this article. Thus, while EI may appear to be more of a psycholinguistic manifestation than a direct test of speaking, it appearsto tap into the same linguistic system that is used for oral communication.

EI is an attractive option for language assessment because it is relatively fast, economical, and effective when compared to traditional proficiency assessments. In addition, because the test is concerned only with the accuracy of the repetition, the scoring procedures for EI assessments are simpler than traditional oral profi ciency tests: Scorers must simply determine whether or not the measured unit (typically a syllable) was produced correctly. Because of this simplicity, many researchers have found success in employing automated speech recognition technology to score EI tests (Cook, McGhee, & Lonsdale, 2011; Graham, Lonsdale, Kennington, Johnson, & McGhee, 2008)- rendering the test even more economical and attractive.

However, the success of an EI instrument as a proficiency measure based on the ACTFL standards, instead of the more common norm-referenced assessments, depends heavily on crafting or choosing items that appropriately reflect the guidelines and discriminate among the proficiency levels of the test-takers (ACTFL, 2012). The question, then, of how the behavioral language sample is operationalized becomes important. Researchers have found that some factors, including item length as measured in number of syllables, grammatical complexity, and lexical complexity can be used to align EI item difficulty with proficiency scales such as the ACTFL scale (Burdis, 2014). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.