Academic journal article Attention, Perception and Psychophysics

Crossmodal Interactions in the Perception of Expressivity in Musical Performance

Academic journal article Attention, Perception and Psychophysics

Crossmodal Interactions in the Perception of Expressivity in Musical Performance

Article excerpt

Published online: 15 November 2013

# Psychonomic Society, Inc. 2013

Abstract In musical performance, bodily gestures play an important role in communicating expressive intentions to audiences. Although previous studies have demonstrated that visual information can have an effect on the perceived expressivity of musical performances, the investigation of audiovisual interactions has been held back by the technical difficulties associated with the generation of controlled, mismatching stimuli.With the present study, we aimed to address this issue by utilizing a novel method in order to generate controlled, balanced stimuli that comprised both matching and mismatching bimodal combinations of different expressive intentions. The aim of Experiment 1 was to investigate the relative contributions of auditory and visual kinematic cues in the perceived expressivity of piano performances, and in Experiment 2 we explored possible crossmodal interactions in the perception of auditory and visual expressivity. The results revealed that although both auditory and visual kinematic cues contribute significantly to the perception of overall expressivity, the effect of visual kinematic cues appears to be somewhat stronger. These results also provide preliminary evidence of crossmodal interactions in the perception of auditory and visual expressivity. In certain performance conditions, visual cues had an effect on the ratings of auditory expressivity, and auditory cues had a small effect on the ratings of visual expressivity.

Keywords Crossmodal interaction . Multisensory integration . Music cognition . Performance . Expressivity . Gesture

Gestures constitute an integral part of human communication. They can facilitate the comprehension of speech, can convey meaning in their own right (e.g., Goldin-Meadow, 1999), and can have an effect on how the meaning of accompanying speech is interpreted (McNeill, Cassell, & McCullough, 1994). The importance of bodily gestures is also one of the parallels between speech and another universal form of human expression and communication- namely, music (e.g., Davidson, 2005). In musical performances, bodily gestures can successfully communicate a range of meaningful infor- mation to audiences and co-performers, including emotional expression (Dahl & Friberg, 2007; Davidson, 1993, 1994), musical structure (Juchniewicz, 2008; Vines, Krumhansl, Wanderley, & Levitin, 2006), and musical ideas and timing (Goebl & Palmer, 2009; Williamon & Davidson, 2002). A recent study even showed that visual kinematic cues are more reliable than auditory cues in the prediction of piano compe- tition winners (Tsay, 2013). Music, like speech, is an inher- ently multisensory phenomenon, comprising auditory, visual, and somatosensory components. Since audio recordings of music have only been commercially available since the early 20th century, the artificial separation of the auditory compo- nent of music from its original multisensory source is a rela- tively recent phenomenon. Yet studies of music perception and cognition have chiefly tended to focus on investigating music as a unisensory, auditory phenomenon.

Music has the capacity to communicate powerful emotional meanings (see Juslin & Laukka, 2003,forareview).Itisgener- ally accepted that music can both express emotions and induce emotional responses in listeners (e.g., Gabrielsson, 2001-2002; Juslin & Sloboda, 2010). The emotional meanings communicat- ed by a musical performance can be successfully decoded by listeners, regardless of their level of musical expertise (Juslin, 1997, 2005). This consistency in terms of the accuracy of decoding emotional meaning may, at least in part, be explained by the similarities between nonverbal, acoustic features of emo- tional speech and the acoustic cues used to communicate emo- tional meaning in music (see Juslin & Laukka, 2003,fora review). Emotional expression is closely related to variations in performance features, which differentiate performances of the same notated work, articulate musical structure (Clarke, 1988), communicate emotional meaning (see Juslin, 2001,forareview), convey a sense of biological motion, and relate to stylistic variation (i. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.