RetailNet: A Deep Learning Approach for People Counting and Hot Spots Detection in Retail Stores

Customer behavior analysis is an essential issue for retailers, allowing for optimized store performance, enhanced customer experience, reduced operational costs, and consequently higher profitability. Nevertheless, not much attention has been given to computer vision approaches to automatically extract relevant information from images that could be of great value to retailers. In this paper, we present a low-cost deep learning approach to estimate the number of people in retail stores in real-time and to detect and visualize hot spots. For this purpose, only an inexpensive RGB camera, such as a surveillance camera, is required. To solve the people counting problem, we employ a supervised learning approach based on a Convolutional Neural Network (CNN) regression model. We also present a four channel image representation named RGBP image, composed of the conventional RGB image and an extra binary image P representing whether there is a visible person in each pixel of the image. To extract the latter information, we developed a foreground/background detection method that considers the peculiarities of people behavior in retail stores. The P image is also exploited to detect the hot spots of the store, which can later be visually analyzed. Several experiments were conducted to validate, evaluate and compare our approach using a dataset comprised of videos that were collected from a surveillance camera placed in a real shoe retail store. Results revealed that our approach is sufficiently robust to be used in real world situations and outperforms straightforward CNN approaches.