Academic journal article Psychonomic Bulletin & Review

Looking at the Rope When Looking for the Snake: Conceptually Mediated Eye Movements during Spoken-Word Recognition

Academic journal article Psychonomic Bulletin & Review

Looking at the Rope When Looking for the Snake: Conceptually Mediated Eye Movements during Spoken-Word Recognition

Article excerpt

Participants' eye movements to four objects displayed on a computer screen were monitored as the participants clicked on the object named in a spoken instruction. The display contained pictures of the referent (e.g., a snake), a competitor that shared features with the visual representation associated with the referent's concept (e.g., a rope), and two distractor objects (e.g., a couch and an umbrella). As the first sounds of the referent's name were heard, the participants were more likely to fixate the visual competitor than to fixate either of the distractor objects. Moreover, this effect was not modulated by the visual similarity between the referent and competitor pictures, independently estimated in a visual similarity rating task. Because the name of the visual competitor did not overlap with the phonetic input, eye movements reflected word-object matching at the level of lexically activated perceptual features and not merely at the level of preactivated sound forms.

Recent years have seen growing interest in the interface between visual perception, language, and action (e.g., Henderson & Ferreira, 2004; Spivey, Tyler, Eberhard, & Tanenhaus, 2001; Zelinsky & Murphy, 2000). The fact that visual perception can influence and be influenced by concurrent linguistic input has prompted the emergence of a new paradigm for studying spoken-language comprehension. In the visual-world paradigm, pioneered by Cooper (1974) and further developed by Tanenhaus, SpiveyKnowlton, Eberhard, and Sedivy (1995), language processing is studied by examining participants' eye gaze to objects in a display as a spoken utterance, referring to one or more objects, is heard (see also Altmann & Kamide, 1999). For example, the instruction "pick up the candle" may be heard concurrently with the presentation of a display containing a candle, candy, pear, and bottle. The probability of fixating each object over time, as linguistic information unfolds, is taken to reflect the listener's developing interpretation of the linguistic input. This assumption relies on people directing their attention-and most often their gaze-to what is being referred to in the utterance, when that referent is visually available. Saccadic eye movements are fast and ballistic; thus, they provide a real-time measure of linguistic processing. Moreover, time course can be assessed without interrupting the speech stream, and the listener's interpretation can be inferred without requiring a metalinguistic decision. Thus, the visual-world paradigm has strengths that complement more traditional methods used to study speech perception.

The relationship between eye movements and spokenlanguage processing has perhaps been demonstrated most convincingly in studies of lexical processing where fixations reflect the dynamics of lexical activation (e.g., Allopenna, Magnuson, & Tanenhaus, 1998; Dahan, Magnuson, & Tanenhaus, 2001 ; Dahan, Magnuson, Tanenhaus, & Hogan, 2001 ; McMurray, Tanenhaus, & Aslin, 2002). The acoustic/phonetic goodness of fit between the speech signal and the names associated with the displayed pictures is a major factor in predicting observed fixations. For example, the probability of fixating a pictured object over time reflects the overlap between the spoken input and the name of the object (Allopenna et al., 1998). As the first sounds of the name unfold (e.g., can from candle), the probability of fixating objects with names consistent with the input (e.g., candle and candy) increases, whereas the probability of fixating objects with inconsistent names (e.g., pear and bottle) decreases. Moreover, lexical frequency modulates the probability of fixating pictured objects; fixations to objects with frequent names increase faster than do fixations to objects with less frequent names (Dahan, Magnuson, & Tanenhaus, 2001; Magnuson, Tanenhaus, Aslin, & Dahan, 2003).

Allopenna et al. (1998) formalized a linking hypothesis relating fixations to the underlying cognitive processes. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.