Academic journal article Psychonomic Bulletin & Review

The Eyes Grasp, the Hands See: Metric Category Knowledge Transfers between Vision and Touch

Academic journal article Psychonomic Bulletin & Review

The Eyes Grasp, the Hands See: Metric Category Knowledge Transfers between Vision and Touch

Article excerpt

Published online: 4 December 2013

© Psychonomic Society, Inc. 2013

Abstract Categorization of seen objects is often determined by the shapes of objects. However, shape is not exclusive to the visual modality: The haptic system also is expert at identifying shapes. Hence, an important question for understanding shape processing is whether humans store separate modality-dependent shape representations, or whether information is integrated into one multisensory representation. To answer this question, we created a metric space of computer-generated novel objects varying in shape. These objects were then printed using a 3-D printer, to generate tangible stimuli. In a categorization experiment, participants first explored the objects visually and haptically. We found that both modalities led to highly similar categorization behavior. Next, participants were trained either visually or haptically on shape categories within the metric space. As expected, visual training increased visual performance, and haptic training increased haptic performance. Importantly, however, we found that visual training also improved haptic performance, and vice versa. Two additional experiments showed that the location of the categorical boundary in the metric space also transferred across modalities, as did heightened discriminability of objects adjacent to the boundary. This observed transfer of metric category knowledge across modalities indicates that visual and haptic forms of shape information are integrated into a shared multisensory representation.

Keywords Shape · Object categorization · Vision · Haptics · Categorization · Multisensory representations

Humans learn about the world by interacting with it, and much of this interaction is mitigated through the sense of touch of our hands. Starting with active exploration of objects in in- fants, up to the precise manipulation skills required by sur- geons, the haptic modality allows us to gather information about an object's shape, texture, softness, weight, temperature, and other material properties. Many of these fundamental object properties, such as temperature and weight, are not or are only indirectly accessible to the other modalities-for example, vision-highlighting the importance of interacting with the world through haptics for developing perceptual and precision skills. In contrast, other object properties, such as shape and texture, are readily accessible to both the visual and haptic modalities. Of these two properties, shape has been found to play a crucial role in object identification and cate- gorization for the visual (e.g., Rosch, Mervis, Gray, Johnson, & Boyes-Braem, 1976) and haptic (e.g., Klatzky, Lederman, &Metzger, 1985) modalities. Since shape can be perceived visually and haptically, the question arises whether shape representations integrate sensory inputs from both modalities, or whether the human brain stores two separate, modality- dependent shape representations.

The idea of common representations is supported by the cross-modal priming observed between vision and haptics (Bushnell & Baxt, 1999; Reales & Ballesteros, 1999). Similarly, several previous studies investigating the perceptual spaces of complex, parametrically defined objects (Cooke, Jäkel, Wallraven, & Bülthoff, 2007; Gaissert, Wallraven, & Bülthoff, 2010) and natural objects (Gaissert & Wallraven, 2011)have revealed high similarity between the visual and haptic perceptual spaces, suggesting similar processing of shape. If one multisen- sory representation is formed, however, one might expect cross- modal shape comparisons to be equivalent in performance to unimodal shape comparisons. In this context, Norman, Norman, Clayton, Lianekhammy, and Zielke (2004) reported high accu- racy but also significant performance differences between cross- modal and unimodal shape comparisons, leading them to con- clude that shape representations are functionally overlapping but distinguishable. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.