Fingertip Interaction Metrics Correlate with Visual and Haptic Perception of Real Surfaces

Yasemin Vardar, Christian Wallraven, Katherine J. Kuchenbecker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

25 Citations (Scopus)


Both vision and touch contribute to the perception of real surfaces. Although there have been many studies on the individual contributions of each sense, it is still unclear how each modality's information is processed and integrated. To fill this gap, we investigated the similarity of visual and haptic perceptual spaces, as well as how well they each correlate with fingertip interaction metrics. Twenty participants interacted with ten different real surfaces from the Penn Haptic Texture Toolkit by either looking at or touching them and judged their similarity in pairs. By analyzing the resulting similarity ratings using non-metric multi-dimensional scaling (NMDS), we found that surfaces are similarly organized within the three-dimensional perceptual spaces of both modalities. Also, between-participant correlations were significantly higher in the haptic condition. In a separate experiment, we obtained the contact forces and accelerations acting on one finger interacting with each surface in a controlled way. We analyzed the collected fingertip interaction data in both the time and frequency domains. Our results suggest that the three perceptual dimensions for each modality can be represented by roughness/smoothness, hardness/softness, and friction, and that these dimensions can be estimated by surface vibration power, tap spectral centroid, and kinetic friction coefficient, respectively.

Original languageEnglish
Title of host publication2019 IEEE World Haptics Conference, WHC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781538694619
Publication statusPublished - 2019 Jul
Event2019 IEEE World Haptics Conference, WHC 2019 - Tokyo, Japan
Duration: 2019 Jul 92019 Jul 12

Publication series

Name2019 IEEE World Haptics Conference, WHC 2019


Conference2019 IEEE World Haptics Conference, WHC 2019

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Biomedical Engineering
  • Sensory Systems
  • Human Factors and Ergonomics


Dive into the research topics of 'Fingertip Interaction Metrics Correlate with Visual and Haptic Perception of Real Surfaces'. Together they form a unique fingerprint.

Cite this