Academic journal article Psychological Test and Assessment Modeling

Developing and Validating an Academic Listening Questionnaire

Academic journal article Psychological Test and Assessment Modeling

Developing and Validating an Academic Listening Questionnaire

Article excerpt


This article reports on the development and administration of the Academic Listening Self-rating Questionnaire (ALSA). The ALSA was developed on the basis of a proposed model of academic listening comprising six related components. The researchers operationalized the model, subjected items to iterative rounds of content analysis, and administered the finalized questionnaire to inter- national ESL (English as a second language) students in Malaysian and Australian universities. Structural equation modeling and rating scale modeling of data provided content-related, substan- tive, and structural validity evidence for the instrument. The researchers explain the utility of the questionnaire for educational and assessment purposes.

Key words: academic listening, language testing, Rating Scale Model, structural equation modeling


Self-rating, a process by which students systematically appraise their own skills and abilities, has attracted significant attention among researchers as an effective tool in language and educational training and assessment (Adams & King, 1995; Brantmeier, 2005, 2006; Cameron, 1990; Heilenmann, 1990; Jafarpour, 1991; Little, 2005; Mowl & Pain, 1995; Orsmond, Merry, & Reiling, 1997; Rivers, 2001; Rolfe, 1990; Ross, 1998; Shore, Shore, & Thornton III, 1992; Stefani, 1994; Sullivan & Hall, 1997). Researchers report it to be a rigorous method of improving language learners' awareness of their own weaknesses and strengths (Ekbatani, 2000), a useful supplement to teacher evaluations (Nunan, 1988), and a way to help teachers understand language learners' self- perceptions, which can direct their teaching.

Educational assessment institutions have begun to use self-rating as an important process in language learning. Little (2005, p. 321) reported that the policies of the Common European Framework of Reference for Languages (CEFR) and the European Language Portfolio (ELP) have shifted toward a more learner-centered learning and assessment paradigm which provides independence to language learners and generates an educa- tional context in which learners "take full account" of their own assessment. Little advo- cated establishing self-rating procedures that "bring the learning process into a closer and more productive relation to tests and examinations than has traditionally been the case" (Little, 2005, p. 324), and using these procedures in high-stakes assessment.

While numerous research studies have examined the principle of self-rating, analysis of its efficacy in academic contexts (e.g., universities and colleges) has been limited. The present study focuses on listening comprehension assessment, an area where self-rating is largely unexplored. Researchers have generated a few self-rating instruments for listen- ing tests (see, for example, Sawaki & Nissan, 2009; Ford & Wolvin, 1992, 1993; Ford, Wolvin, & Sungeun, 2000), but their efficacy, and the utility of self-rating in listening comprehension generally, has not been examined.

We present an English academic listening self-rating questionnaire. This questionnaire is designed to help students in supplementary and academic English language courses im- prove their academic listening performance and become more autonomous and aware of their proficiency level. The questionnaire is based on an exploratory model of integrated academic listening macroskills (see Figure 1). Since multiple studies have reported weak or even negative correlations between student scores on the listening section of the Inter- national English Language Testing System (IELTS(TM)) (a popular English proficiency test for university admissions) and their subsequent grade point averages (see Aryadoust, 2011a), the model we propose is also intended to help accurately represent and assess academic listening. The study provides content-referenced, substantive, and structural validity evidence for the questionnaire. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.