Academic journal article Attention, Perception and Psychophysics

Quantifying Temporal Ventriloquism in Audiovisual Synchrony Perception

Academic journal article Attention, Perception and Psychophysics

Quantifying Temporal Ventriloquism in Audiovisual Synchrony Perception

Article excerpt

Published online: 19 July 2013

© Psychonomic Society, Inc. 2013

Abstract The integration of visual and auditory inputs in the human brain works properly only if the components are perceived in close temporal proximity. In the present study, we quantified cross-modal interactions in the human brain for audiovisual stimuli with temporal asynchronies, using a paradigm from rhythm perception. In this method, participants had to align the temporal position of a target in a rhythmic sequence of four markers. In the first experiment, target and markers consisted of a visual flash or an auditory noise burst, and all four combinations of target and marker modalities were tested. In the same-modality conditions, no temporal biases and a high precision of the adjusted temporal position of the target were observed. In the different-modality conditions, we found a systematic temporal bias of 25-30 ms. In the second part of the first and in a second experiment, we tested conditions in which audiovisual markers with different stimulus onset asynchronies (SOAs) between the two components and a visual target were used to quantify temporal ventriloquism. The adjusted target positions varied by up to about 50 ms and depended in a systematic way on the SOA and its proximity to the point of subjective synchrony. These data allowed testing different quantitative models. The most satisfying model, based on work by Maij, Brenner, and Smeets (Journal of Neurophysiology 102, 490-495, 2009), linked temporal ventriloquism and the percept of synchrony and was capable of adequately describing the results from the present study, as well as those of some earlier experiments.

Keywords Audiovisual synchrony · Temporal ventriloquism · Temporal alignment

(ProQuest: ... denotes formula omitted.)

Perceiving temporal discrepancies between the various sen- sory components of specific events rarely happens in daily life. However, everyone knows that in technology-driven events-that is, speech on television or communication by webcams-a temporal conflict can occur when the sound is not exactly aligned with the accompanying visual informa- tion. Furthermore, the asynchrony is noticed only when one of the sensory inputs is delayed by an amount that is larger than the temporal delay that the human brain can integrate. This window of temporal integration depends on many factors-that is, context and stimulus identity (Van Eijk, Kohlrausch, Juola, & Van de Par, 2008, 2010). The fact that different propagation speeds for light and sound for distal stimuli are combined with different neural conduction times for proximal stimuli means that the brain must have some tolerance for audiovisual (AV) temporal discrepancies (Vroomen & Keetels, 2010).

Multiple studies focusing on temporal conflicts be- tween visual and auditory information have concluded that in the temporal domain, audition is the more dominant sen- sory modality (Aschersleben & Bertelson, 2003;Bertelson & Aschersleben, 2003; Morein-Zamir, Soto-Faraco, & Kingstone, 2003). Shams, Kamitani, and Shimojo (2000, 2002) studied the illusory flash effect, in which multiple auditory beeps can change the perception of a single flash into two or more flashes. For periodically modulated auditory and visual stimuli, it has been long known that, in the case of temporal-rate disparities, the perceived rate of visual modulation is strongly influenced by the rate of auditory modulation, while the reverse effect is nearly absent (for an early demonstration, see Gebhard & Mowbray, 1959). More recent research on this topic has shown that the amount of cross-modal interaction in perceived rate can be controlled by adjusting the within-modality rate discrimination sensitivity-for instance, by changing the depth of modula- tion (Roach, Heron, & McGraw, 2006). Also, the perceived time of occurrence of a visual signal can be influenced by asynchronous sounds (Bertelson & Aschersleben, 2003; Morein-Zamir et al. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.