Classification and tracking of items on a moving conveyor belt using convolutional networks and image processing

The prolonged waiting time at supermarket checkout lines poses a significant challenge to the shopping experience, impacting customer satisfaction and operational efficiency. This paper presents a prototype that addresses this issue by utilizing computer vision and deep learning. The model, incorporating convolutional neural networks such as YOLO v4 tiny and YOLO v5 small, along with tools like OpenCV and Roboflow for dataset management, achieves a remarkable 98% mean average precision for two-class detection. It efficiently detects, classifies, tracks, and counts items on a mobile supermarket conveyor belt. Additionally, we introduce a versatile framework designed for seamless integration into real-world applications. It comprises a customizable monitoring application and simulator that facilitates synthetic image data generation. Managing diverse items in a supermarket presents a major challenge for data gathering, labeling, and training. In that sense, the importance of customizable monitoring and simulation tools is highlighted, emphasizing their practical role. Our findings demonstrate the feasibility of maintaining a minimal 0% to 2.85% precision tradeoff while using half of the data as synthetic for two-class detection, indicating potential practicality in supermarkets with proper scaling. In summary, this study brings tangible benefits to both customers and retailers, offering a potential to streamline, speed up, and cut costs in the supermarket checkout process.