Texture Classification by Audio-Tactile Crossmodal Congruence

This work presents a large-scale texture classification method using a novel texture feature Projected Spectral Mapping (PSM) based on audio-tactile crossmodal congruence in unconstrained tool-surface interactions. We describe a quick-computable extraction process for PSM from the proposed crossmodal inter-band spectral mapping (IBSM) that relates the frequency components in different bands between the modalities. We conducted a texture classification on the LMT Haptic Texture Dataset with 69 textures in 9 categories to evaluate the PSM feature by both random sampling train-test split and participant-specific cross-validation. Compared to a variety of texture features from previous work, the results showed that our PSM feature reached >74 % classification accuracy by 3 out of 4 classifiers and outperformed all other features with significant improvement.