Academic journal article Australian Journal of Language and Literacy

The Text-Image Interaction and Second Language Learning

Academic journal article Australian Journal of Language and Literacy

The Text-Image Interaction and Second Language Learning

Article excerpt


The aim of this article is to present some of the findings of a research study on the text-image relations in picture books used with children learning English as a Second Language that was undertaken in order to find out how and to what extent the visual images represent the meanings communicated by the verbal text.

The study was carried out within a multi-layered approach that focussed on the generic structure of the stories, the linguistic choices made by the writer, and the visual choices made by the illustrator that interact with the verbal text.

Theoretical background

Kress and van Leeuwen take a fresh look at the question of visual literacy starting from the assumption that `visual communication has its own grammar, that images are amenable to rational accounts and analysis, and that language and visual communication both realise the same fundamental and far-reaching systems of meaning that constitute our culture, each by means of its own specific forms and independently' (1990: p. 4).

By emphasising the independent nature of visual communication, Kress and van Leeuwen move away from the position that Barthes took in his essay `Rhetoric of the Image' (1977), where he argued that because images are polysemous they are dependent on the verbal text which exercises a function of control over the potential meanings of the image. Barthes identified two basic image-text relations: elaboration and relay. In elaboration, the verbal text restates the meanings of the image or vice-versa. In other words, the same meanings are communicated by the verbal code and the visual code. In relay, the verbal text extends the meanings of the image or vice-versa, with word and image in a complementary relation. Although Kress and van Leeuwen aim at reassessing the value of visual communication, they claim that their proposal would not have been possible without recent advances in linguistics. In this respect, their model of visual assessment seems promising because they choose to begin with a discussion of language and the different layers of meanings conveyed by language before moving to a discussion of the meanings conveyed by images. In spite of the fact that these researchers point out that their theory of visual communication is in its beginning stages, it offers a new perspective to the study of picture books by encouraging a linguistic-oriented approach to the analysis of children's literature which, despite its widely recognised influence on children's social, cognitive and linguistic development, has received (as Knowles and Malmkjaer [1996] observe) little linguistic analysis.

Written stories that are used in second language learning have the undeniable value of providing the context through which children can acquire the new language by being exposed to the sentence patterns, collocations and rhythms of the target language. But at the same time, the second language can be a barrier that prevents the young second language learners from getting into the world of a story. The presence of pictures illustrating the written narrative can, however, facilitate the decoding process by making the language of the story not only meaningful but also memorable.

Toolan observes that:

the business of experiencing and understanding the implications of text-scene matching, which all illustrated stories nurture, is a crucial step to the more decontextualised children's story, the one with text alone, where the child is required to produce in her own mind, using her imaginative resources, satisfying mental pictures of what is going on.

(1988: p. 211)

Second language teachers should be prepared to guide and stimulate the process of text-scene matching so that children not only construct mental representations of the stories they read but also develop their linguistic competence in the second language they are learning. When text and image are simultaneous input, the image will surely first engage their attention. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.