Deep Learning based Litter Identification and Adaptive Cleaning using Self-reconfigurable Pavement Sweeping Robot

Pavement sweeping, which is primarily labor-intensive, is essential to keep it clean and hygienic for use. Humans play the role of identifying the litter to pick and adjust the vacuum suction power. In this paper, we propose a framework that consists of two layers, namely, a) the first method is to identify commonly found litter on pavements, b) secondly, an adaptive vacuum suction scheme based on fuzzy logic is implemented for more efficient pick up of the identified litter. Semantic segmentation using Convolution Neural Network (CNN) SegNet was adopted to segment the pavement region from other objects. Then, the Deep Convolutional Neural Network (DCNN) based object detection is used to detect pavement litter. Afterward, the calibrated vacuum suction as per identified litter was selected based on fuzzy-based adaptive actuation. Further, the proposed framework’s efficacy is successfully tested on a self-reconfigurable pavement sweeping robot named Panthera. The inspection framework was configured in Jetson Nano Nvidia GPU and took approximately 132.2 milliseconds to detect litter. In the experiment conducted, there is a 38.5 % improvement in energy consumption for the pavement cleaning task using a depth-based vision system and a vacuum suction motor and can be used in runtime.