In-Hand Small-Object Counting from Tactile Sensor Arrays Installed on Soft Fingertips

In object picking, knowing the state of picked up objects is very important to conduct succeeding tasks surely. Especially, the ability to count objects in hand is crucial to judge whether the previous picking action was successful or not. This work seeks to endow such ability to a robot manipulator. Vision-based methods cannot be relied on to count objects in hand due to the occlusion problem, especially when dealing with objects smaller than one centimeter like small screws. Hence, number estimation should be conducted from tactile sensor information. However, compact pressure-based tactile sensor arrays can not take fine outlines of such small objects because sensor element size is not small enough, meaning that a simple rule-based approach is not feasible. Furthermore, the tactile sensor array fixed on a rigid plane surface can only contact protruding parts of in-hand objects; thus, the simple installation of tactile sensor arrays on a manipulator surface is insufficient for counting objects. Therefore, in this work, we propose 1) a number estimation method which uses a convolution neural network and 2) to cover a tactile sensor array with soft material to enrich tactile information to improve estimation accuracy. We validated the proposed method using data collected by a simple gripper and achieved 89% accuracy in estimating the small screw number in the gripper.