Academic journal article English Language Teaching

Comparative Coh-Metrix Analysis of Reading Comprehension Texts: Unified (Russian) State Exam in English vs Cambridge First Certificate in English

Academic journal article English Language Teaching

Comparative Coh-Metrix Analysis of Reading Comprehension Texts: Unified (Russian) State Exam in English vs Cambridge First Certificate in English

Article excerpt

Abstract

The article summarizes the results of the comparative study of Reading comprehension texts used in B2 level tests: Unified (Russia) State Exam in English (EGE) and Cambridge First Certificate in English (FCE). The research conducted was mainly focused on six parameters measured with the Coh-Metrix, a computational tool producing indices of the linguistic and discourse representations of a text: narrativity, syntactic simplicity, word concreteness, referential cohesion, deep cohesion, Flesh Reading Ease. The research shows that the complexity of EGE texts caused by lower than in FCE texts cohesion is balanced with a simpler than in FCE texts syntax and higher narrativity thus resulting in about the same text complexity of the two sets of texts studied. EGE and FCE texts demonstrate correspondence to grade six and very similar Means of Flesh Reading Ease (FCE Mean is 71.06; EGE Mean is 78.25) which fit the band FAIRLY EASY.

Keywords: Unified (Russia) State Exam (EGE), Cambridge First Certificate in English (FCE), Coh-Metrix, Flesh Reading Ease, narrativity, syntactic simplicity, word concreteness, referential cohesion, deep cohesion

1. Introduction

Reading as a key element in EFL testing research is typically studied with the focus on one of the three interacting factors that make the reading comprehension process more or less challenging: reader characteristics, text characteristics, and characteristics of the activity of reading.

Characteristics of the texts used in standardized tests remain a fairly neglected area in Language assessment research and publication. Responding to this effective gap in the literature, the present research is geared towards the lack of a systematic approach to parameters of the texts used in language assessment.

The main objective of the research presented is to find out in which way the text characteristics used in reading multiple choice parts of Unified (Russian) State Exam in English (EGE) differ from those of Cambridge First Certificate in English (FCE) by comparing their narrativity, syntactic simplicity, word concreteness, referential cohesion, deep cohesion, Flesh Reading Ease. Our side objective in highlighting such differences is to provide textual information that may facilitate the work of non-native English item-writers and offer some direction for reading materials selection and modification.

2. Literature Review

The text characteristics include a wide variety of quantitative and qualitative parameters: length of the text, sentence and word length, frequency of unfamiliar words, vocabulary, language structures, text structure, genre, and background knowledge assumed in the text that students read. Although educators sometimes speak of students' "reading level," a given student might be quite successful at reading a text that is in a familiar format and about a favorite topic, but then struggle to read an academic text, even if it is at the same level as measured by a readability formula. Research shows that test takers considered 'below level' based on academic assessments can demonstrate high-level comprehension of sophisticated texts selected in other contexts (Moje, 2000).

2.1 Quantitative Text Characteristics

Modern quantitative measures calculate text complexity with a number of Readability formulas, more than 40 of which have been developed over the years (Klare, 1974-1975). Readability formulas assign a grade level equivalent or Lexile levels for the texts. The quantitative text parameters theories are based on a number of assumptions, the most common of which, according to Zipf (1949) are as follows: texts with longer words and lengthier sentences are more difficult to read. Longer words tend to be less frequent in the discourse, and infrequent words take more time to access and interpret during reading (Just & Carpenter, 1980). Longer sentences tend to place more demands on working memory and are therefore more difficult (Graesser et al. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.