Academic journal article Psychonomic Bulletin & Review

The Role of Cross-Modal Associations in Statistical Learning

Academic journal article Psychonomic Bulletin & Review

The Role of Cross-Modal Associations in Statistical Learning

Article excerpt

Published online: 29 May 2013

© Psychonomic Society, Inc. 2013

Abstract Our environment is richly structured, with objects producing correlated information within and across sensory modalities. A prominent challenge faced by our perceptual system is to learn such regularities. Here, we examined statistical learning and addressed learners' ability to track transitional probabilities between elements in the auditory and visual modalities. Specifically, we investigated whether cross-modal information affects statistical learning within a single modality. Participants were familiarized with a statistically structured modality (e.g., either audition or vision) accompanied by different types of cues in a second modality (e.g., vision or audition). The results revealed that statistical learning within either modality is affected by cross-modal information, with learning being enhanced or reduced according to the type of cue provided in the second modality.

Keywords Statistical learning * Cross-modal associations * Grouping

Introduction

Our environment is richly structured, with objects producing correlated information across different sensory modalities. For example, a roaring bear will be perceived as "big" by both our auditory and visual modalities. A prominent challenge faced by our perceptual system is to learn such regularities. Indeed, numerous studies have revealed both infants' and adults' exceptional ability to learn various types of regularities (e.g., Bahrick & Lickliter, 2000; Cohen, Ivry, & Keele, 1990;Ernst, 2007; Reber, 1967). During our initial development, cross-modal information benefits certain forms of learning (Gibson, 1969), and this cross-modal advantage is thought to be pre- served throughout our lives (Kim, Seitz, & Shams, 2008). Here we focus on a paradigm termed statistical learning,in which learners track transitional probabilities (TPs) between elements in the auditory and visual modalities (Fiser & Aslin, 2002a, 2002b; Saffran, Aslin, & Newport, 1996). For exam- ple, after listening to a concatenated speech stream, with certain syllables forming reoccurring "words," infants are able to discriminate between those "words" and the sequences of syllables that never appeared successively (Saffran et al., 1996). Similar findings were obtained in the visual modality (Fiser & Aslin, 2002a, 2002b). However, most research to date has focused on statistical learning within single modali- ties (hence, "unimodal" statistical learning). Here, we will ask whether cross-modal information affects statistical learning, and whether learning is enhanced or reduced according to the type of association between modalities.

Cross-modal information has been found to play a prominent role in different perceptual and cognitive processes. For example, participants are faster at localizing auditory-visual targets than at localizing either auditory or visual targets alone (Hughes, Reuter- Lorenz, Nozawa, & Fendrich, 1994). Information presented from two modalities is also remembered better: Following a period of encoding in which participants were presented with either pic- ture-sound pairs or pictures alone, they subsequently showed better retrieval of the pictures that had formerly been paired with sounds, as compared to the pictures that had not (Lehmann & Murray, 2005).

A cross-modal benefit is observed for various types of learning, as well, for both infants and adults. Gibson (1969) long ago suggested that during early development redundant information across two or more modalities (properties that are not specific to one modality, but are redundant across multiple modalities, such as size or rhythm) is "foregrounded" from the environment by having attentional precedence over modality- specific information. In support of this claim, it was found that five-month-old infants learned an abstract rule (e.g., ABA) or rhythm inferred from both auditory and visual stimuli, but they did not learn the same underlying structure when it was presented through stimuli from one modality alone (Bahrick & Lickliter, 2000; Frank, Slemmer, Marcus, & Johnson, 2009). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.