Robust Estimation of Product Amount on Store Shelves from a Surveillance Camera for Improving On-Shelf Availability

This paper proposes a method to robustly estimate product amount on store shelves from a surveillance camera for improving on-shelf availability. We focus on changes of products on the shelves such as “product taken (decrease)” and “product replenished/returned (increase)”, and compute product amount by accurately accumulating them. The proposed method first detects change regions of products on the shelves in an image using background subtraction followed by moving object removal. The detected change regions are then classified into several classes representing the actual changes on the shelves such as “product taken” by using convolutional neural networks. Finally, the changes of products on the shelves are accumulated using classification results, and product amount on the shelves visible in the image is computed as on-shelf availability. Experimental results using two videos captured in a real store show that our method achieves success rate of 89.6% for on-shelf availability when an error margin is within one product. With high accuracy, store clerks can keep high on-shelf availability, enabling the improvement of business profit in retail stores.