Data-driven thermal recognition of contact with people and objects

Many tactile sensors can readily detect physical contact with an object, but tactile recognition of the type of object remains challenging. In this paper, we provide evidence that data-driven thermal tactile sensing can be used to recognize contact with people and objects in real-world settings. We created a portable handheld device with three tactile sensing modalities: a heat-transfer sensor that is actively heated, a small thermally-isolated temperature sensor, and a force sensor to detect the onset of contact. Using this device, we collected data from contact with the arms of 10 people (3 locations on the right arm) and contact with 80 objects relevant to robotic assistance (8 object types in 10 residential bathrooms). We then used support vector machines (SVMs) to perform binary classifications relevant to assistive robots. When classifying contact as person vs. object, classifiers that only used the temperature sensor performed best (average accuracy of 98.75% for 3.65s of contact, 93.13% for 1.0s, and 82.13% for 0.5s). When classifying contact into two task-relevant object types (e.g., towel vs. towel rack), classifiers that used the heat-transfer sensor together with the temperature sensor performed best. Performance was good when generalizing to new contact locations in the same environment (average accuracy of 92.14% for 3.65s of contact, 91.43% for 1.0s, and 84.29% for 0.5s), but weaker when generalizing to new environments (average accuracy of 84% for 3.65s of contact, 71% for 1.0s, and 65% for 0.5s).