Academic journal article Trames

Natural Speaking and How to Assess It

Academic journal article Trames

Natural Speaking and How to Assess It

Article excerpt

1. Introduction

In Europe, the standard set of levels of language proficiency (CEFR) has been widely accepted as a common standard to help linguistic communities establish within and between themselves (a) language learning objectives for learners needing to manage in typical work, social, personal, or educational situations (the action-oriented approach), and (b) proficiency exams to measure how learners cope with these situations.

As test takers' chances of getting a job in future or continuing their studies depend on the results of any official exam (so-called high-stakes exams), language testing should ideally be balanced and fair. At the same time, because there are no benchmarks to measure how native speakers cope with the situations as compared to the L2-speakers of the language (1), it is easy for the assessors to give too much credit to standard (i.e. normative) language use or to features of speaking or writing that are considered to be correct in school-grammar, but never measured to be characteristic of modern natural speech. Our goal is to measure some parameters of real-life use of the educated Estonian language in order to put these aspects in the right proportion. As some features of L2 use may be irrelevant in managing particular situations, and should not be taken into account in testing under the action-oriented approach, we also collected L2 data and compared it to L1 data.

Until the autumn of 2008, Estonian language proficiency was tested nationally on three levels. After this date the proficiency exams of the Council of Europe were adopted (Table 1). To harmonise national and CE language exams, new guidelines for assessing writing and speaking skills were needed.

This article looks at the speaking proficiency of the proficient or effective user, the level where all essential language skills and competences can be tested (see CEFR, or in Estonian, Kerge 2008).

Speaking involves two essential skills: oral interaction (dialogue) and oral presentation (monologue). Both skills were tested in high-level language proficiency exams until 2008, and are now tested in C1-level exams. In this study we look at both monologues and dialogues. We believe that the test taker's performance should not be compared to the standard or officially correct usage, sometimes unattainable even for a native L1 linguist, but to natural usage, which focuses on the message, undisturbed by the sound of speaking or the choice, form, order and binding of lexical units (see also Ratcliff et al. 2002).

We take natural language to mean the language that is usually spoken and written by a university-educated non-linguist. The language use of such a person can be considered the reference model of natural language. Our research task is to establish which aspects are important in defining natural speech, and which aspects should be taken into account when assessing speaking skills at a C1-level exam. (Our research questions are presented below under the parameters studied).

Depending on the text type (monologue or dialogue) (2), we will look at the following aspects: (1) lexical richness (the Uber index of the balance of words and tokens) and vocabulary range (proportion of basic vs. rare words); (2) contextuality and formality (F-index relating context-free vs. context-bound vocabulary, inversely proportional to text ambiguity); (3) complicacy of syntax (sentence length and complexity, plus degree of nominalisation); (4) temporal characteristics of the dialogue (culture-specific length of turn, pausing, simultaneous talking); (5) strength and disruptiveness of the L2 foreign accent (relationship between the perceived strength and disruptiveness); (6) aspects of L2 intonation (patterns of rising intonation).

In order to establish a foundation for the L2 proficiency assessment, we will compare L1 and L2 dialogues and monologues produced at language exams, and present comparative data on spontaneous Estonian speech in dialogues. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.