Visual Affordance Guided Tactile Material Recognition for Waste Recycling

Because more and more solid waste is generated, in particular, in cities, the management of solid waste disposal has become a global challenge. A solution is to find an effective way to sort solid waste materials and recycle them into reusable products. In this article, we propose to use a material recognition method with vision-guided tactile to form a robotic system for waste sorting. The vision guidance module integrates an object detector and an affordance network together. It allows the robot to not only detect the desired containers and packaging from an assortment of the waste but also obtain a configuration to grasp the target and actively collect its tactile data. By classifying the object with the tactile data, the robot can sort containers and packaging into their respective categories according to the type of material. Our experimental results demonstrate the effectiveness of the proposed robotic waste sorting system in sorting containers and packaging various types of materials. Note to Practitioners—The management of waste has become a great challenge in environmental protection in the world. We propose a robotic waste sorting system, which utilizes a vision-guided tactile sensing approach to find target waste and sort the waste according to materials. A visual module is used to find the waste of interest. A class-specific affordance map is generated to guide a robotic hand to actively grasp waste and collect the tactile data for material recognition. With the recorded tactile data, the robot can recognize the material of the target waste and sort the waste according to the type of material. The proposed system demonstrates good performance in waste sorting, and it can be easily implemented in practical scenarios. The target material can be generalized to a broader group of objects.