Academic journal article Attention, Perception and Psychophysics

When Emotional Valence Modulates Audiovisual Integration

Academic journal article Attention, Perception and Psychophysics

When Emotional Valence Modulates Audiovisual Integration

Article excerpt

Published online: 24 May 2012

# Psychonomic Society, Inc. 2012

Abstract We constantly integrate the information that is available to our various senses. The extent to which the mechanisms of multisensory integration are subject to the influences of attention, emotion, and/or motivation is currently unknown. The "ventriloquist effect" is widely assumed to be an automatic crossmodal phenomenon, shifting the perceived location of an auditory stimulus toward a concurrently presented visual stimulus. In the present study, we examined whether audiovisual binding, as indicated by the magnitude of the ventriloquist effect, is influenced by threatening auditory stimuli presented prior to the ventriloquist experiment. Syllables spoken in a fearful voice were presented from one of eight loudspeakers, while syllables spoken in a neutral voice were presented from the other seven locations. Subsequently, participants had to localize pure tones while trying to ignore concurrent visual stimuli (both the auditory and the visual stimuli here were emotionally neutral). A reliable ventriloquist effect was observed. The emotional stimulus manipulation resulted in a reduction of the magnitude of the subsequently measured ventriloquist effect in both hemifields, as compared to a control group exposed to a similar attention-capturing, but nonemotional, manipulation. These results suggest that the emotional system is capable of influencing multisensory binding processes that have heretofore been considered automatic.

Keywords Multisensory processing . Spatial localization

We live in a multisensory world, meaning that for most perceptual problems, information from multiple modalities is available at the same time. The question of how to combine multisensory inputs has given rise to a number of theories over the years (e.g., Ernst & Bülthoff, 2004; Welch, 1999; Welch & Warren, 1980). Currently, it is commonly assumed that a relation exists between how appropriate a sensory modality is for a given perceptual task (or, more precisely, how reliable the perceptual input from that sensory channel is) and the degree of influence that this modality has on the ensuing multisensory perceptual estimate. For example, for the task of localizing an audiovisual target, we would expect vision to dominate the multisensory estimate, because the visual system tends to be more reliable than the auditory system for spatial tasks (e.g., Alais & Burr, 2004).

This form of visual dominance sometimes results in illusory percepts such as the "ventriloquist effect" (see Bertelson & de Gelder, 2004, for a review). If participants have to localize tones while trying to ignore concurrently presented, spatially displaced visual stimuli, their responses are usually biased toward the to-be-ignored visual stimuli. Abundant evidence has suggested that ventriloquism is an automatic, genuinely perceptual phenomenon (e.g., Bertelson & Aschersleben, 1998; Bertelson, Pavani, Ladavas, Vroomen, & deGelder, 2000; Bertelson, Vroomen, de Gelder, & Driver, 2000; Bonath et al., 2007; Driver, 1996; Vroomen, Bertelson, & de Gelder, 2001). This assumption is supported by the observation that the audiovisual ventriloquist effect is so robust that it still occurs even if the participants are (or become) aware of the spatial discrepancy between the auditory and visual stimuli (see Radeau & Bertelson, 1974). However, more recently, some authors have questioned the full automaticity of this crossmodal binding effect and have suggested that attention might at least have a modulating influence (Fairhall & Macaluso, 2009; Röder & Büchel, 2009). Here, we tested whether or not crossmodal binding is an automatic process. In order to maximize the likelihood of interfering with a presumably automatic crossmodal binding process, as assessed with the ventriloquist illusion, we used emotional stimuli.

The emotional significance of stimuli has often been found to enhance attentional, perceptual, and cognitive processes (Vuilleumier, 2005). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.