Academic journal article Perception and Psychophysics

Audiovisual Synchrony and Temporal Order Judgments: Effects of Experimental Method and Stimulus Type

Academic journal article Perception and Psychophysics

Audiovisual Synchrony and Temporal Order Judgments: Effects of Experimental Method and Stimulus Type

Article excerpt

When an audio-visual event is perceived in the natural environment, a physical delay will always occur between the arrival of the leading visual component and that of the trailing auditory component. This natural timing relationship suggests that the point of subjective simultaneity (PSS) should occur at an auditory delay greater than or equal to 0 msec. A review of the literature suggests that PSS estimates derived from a temporal order judgment (TOJ) task differ from those derived from a synchrony judgment (SJ) task, with (unnatural) auditory-leading PSS values reported mainly for the TOJ task. We report data from two stimulus types that differed in terms of complexity-namely, (1) a flash and a click and (2) a bouncing ball and an impact sound. The same participants judged the temporal order and synchrony of both stimulus types, using three experimental methods: (1) a TOJ task with two response categories ("audio first" or "video first"), (2) an SJ task with two response categories ("synchronous" or "asynchronous"; SJ2), and (3) an SJ task with three response categories ("audio first," "synchronous," or "video first"; SJ3). Both stimulus types produced correlated PSS estimates with the SJ tasks, but the estimates from the TOJ procedure were uncorrelated with those obtained from the SJ tasks. These results suggest that the SJ task should be preferred over the TOJ task when the primary interest is in perceived audio-visual synchrony.

Most everyday events give rise to both visual and auditory sensations, such as when we listen to someone speaking in front of us or observe a book falling to the floor. Because of the relatively low speed of sound, the auditory component of a perceived event will reach an observer later than the visual component, and this difference increases with physical distance. Thus, when auditory and visual components of a single audio-visual event reach an observer's sensory receptors, there will always be a physical delay between the leading visual component and the following auditory component. At least part of this difference in arrival times can be made up (or even reversed, for proximal stimuli) by the faster processing of auditory information once it arrives at the sensory receptors, since brain activation occurs about 30-50 msec earlier for sounds than for visual stimuli that arrive simultaneously with them (see, e.g., Arrighi, Alais, & Burr, 2006; King & Palmer, 1985; Pöppel, 1988).

As a result of the different propagation speeds of light and sound, observers should be more tolerant of lagging audio than of lagging video when integrating the sensed components of an event. This tolerance can be measured in terms of the milliseconds of auditory delay over which integration into perception of a unitary common event is satisfactorily achieved. By convention (see, e.g., Arrighi et al., 2006; Aschersleben & Müsseier, 1999; Enoki, Washikita, & Yamada, 2006; Vatakis & Spence, 2006a, 2006b; Zampini, Shore, & Spence, 2003a, 2003b), such auditory delays are measured in positive values (with 0 msec indicating physical synchrony of the auditory and visual stimulus components), whereas negative values are used for the far-less-frequent occurrence of an auditory component of some event arriving before its visual counterpart. We can estimate the point of subjective simultaneity (PSS) from the midpoint of the range of delays within which synchrony is perceived. Furthermore, we expect that the PSS should occur near the point of physical synchrony or at some positive audio delay-that is, with the auditory component lagging behind the visual component.

Research in the area of perceived audio-visual synchrony has made use of a wide range of stimulus types (see, e.g., Arrighi et al., 2006; Enoki et al., 2006; Keetels & Vroomen, 2005; Vatakis & Spence, 2006a) and experimental methods (see, e.g., Dixon & Spitz, 1980; Exner, 1875; Vatakis, Navarra, Soto-Faraco, & Spence, 2008; Vroomen, Keetels, de Gelder, & Bertelson, 2004). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.