Vi2TaP: A Cross-Polarization Based Mechanism for Perception Transition in Tactile-Proximity Sensing With Applications to Soft Grippers

Vision-based soft sensors have emerged as a promising solution for multi-modal sensory systems. Rather than relying on complex integrations of numerous specialized sensors, these devices are advantageous in achieving multiple perceptual capabilities with a single device. However, the unsolved bottleneck for existing systems lies in preventing perceptual interference among visual fields and establishing a reliable mechanism for switching between perception domains. In this study, we present ${Vi2TaP}$, a novel mechanism leveraging the cross-polarization phenomenon to shift one perception domain to another in a tactile-proximity multimodal sensing paradigm. The core concept involves two polarizer films placed back-to-back. By adjusting the Plane of Polarization (PoP) between 0 to 90 deg, the camera can either fully open its Field-of-View (FoV) to the external environment for proximity sensing or restrict it to the internal space between the polarizers, tailored for tactile sensing. First implementation of ${Vi2TaP}$ is showcased on a soft sensorized gripper. Additionally, we introduce efficient learning pipelines for both proximity and tactile perception, along with effective strategies for extracting valuable information. The experiment results have demonstrated the advantages of the proposed multi-modal sensing scheme in grasping and manipulating tasks. This mechanism is anticipated to accelerate the development and adoption of multimodal vision-based soft sensors across a wide range of practical applications.