Academic journal article Australian Journal of Language and Literacy

Music, Multiliteracies and Multimodality: Exploring the Book and Movie Versions of Shaun Tan's the Lost Thing

Academic journal article Australian Journal of Language and Literacy

Music, Multiliteracies and Multimodality: Exploring the Book and Movie Versions of Shaun Tan's the Lost Thing

Article excerpt


A substantial and growing number of well-known picture books and illustrated novels have now been adapted as digital animated movie versions, and while some similar stories are now composed for the digital moving image format, such as The Fantastic Flying Books of Mr Morris Lessmore (Joyce, 2011), a number of picture book authors are simultaneously producing book and digital media versions of their stories such as The Heart and the Bottle and the corresponding i-pad app (Jeffers, 2009). The appearance of movie versions of established picture books and illustrated novels are frequently highly celebrated within broad popular culture, as was the case, for example, with the movie of Where the Wild Things Are (Jonze, 2009) from Maurice Sendak's classic picture book (1962), the movie of Fantastic Mr Fox (Anderson, 2009) from the illustrated novel by Roald Dahl (1974), and the movie of The Polar Express (Zemeckis, 2004) from the well-known picture book by Chris Van Allsburg (1985).

In many 'parallel' paper and digital media versions of such literature the story is ostensibly the same, although some, such as The Polar Express are quite different. But for many, at least the events and the characters appear to be the same, with the animated images and language in the movie often not readily discernibly different from the book. However, even subtle changes in language, and different choices in the positioning of images, and in the visual point of view, can construct very different interpretive possibilities for the movie and book versions. In all of these adaptations, of course, music is a fundamental distinguishing feature of the movie versions. However, notwithstanding the significance of music in digital multimodal texts (Noad & Unsworth, 2007) and studies comparing picture books with their animated versions (Bearne, 2003; Parker, 1999, 2001; Unsworth, 2003), it is very difficult to find any discussion of the role of music in these digital multimodal literary narratives. Those who have drawn attention to the multimodal nature of literacy and have advocated a multiliteracies pedagogy, have privileged the role of images in multimodal texts to the virtual exclusion of any consideration of music (Anstey, 2002; Anstey & Bull, 2006; Cope & Kalantzis, 2000; Kist, 2005; Lankshear & Knobel, 2003, 2006; Leu, 2006; Leu, Kinzer, Coiro & Cammack, 2004; Unsworth, 2001). However the new national Australian Curriculum: English (Australian Curriculum Assessment and Reporting Authority, 2010) makes quite explicit reference to the role of music in students' experience of multimodal literary texts. For example, in year three students are expected to ...

Create texts that adapt language features and patterns encountered in literary texts, for example characterisation, rhyme, rhythm, mood, music, sound effects and dialogue ... (and)... multimodal texts that combine visual images, sound effects, music and voice overs to convey settings and events in a fantasy world (ACELT1791).

And in year eight the students are expected to be ...

creating and performing scripts for short plays that make use of the affordances of visual, verbal and additional modes (for example music) to create atmosphere, to deepen interpretation of verbal meaning and to enhance the drama of a performance (ACELT1768)

In order to inform such multimodal literacy practices and their development through multimodal literacy pedagogy, it is essential to develop a literacy research and pedagogic agenda that includes music within the repertoire of multimodal literacy resources (Duncum, 2004).

In this paper we will first briefly indicate the joint influence of traditions of literary appreciation of picture books and functional semiotic accounts of the meaning-making resources of language and images in shaping our approach to explicating the multimodal nature of the interpretive possibilities constructed by language and images in picture books. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.