Customer Behavior Recognition in Retail Store from Surveillance Camera

The analysis of customer behavior from surveillance camera is one of the most important open topics for marketing. We develop a system to recognize different customer behaviors on the front of shelf: no interest, viewing, turning to shelf, touching, picking and returning to shelf and picking and putting into basket, which show customer's increasing interest to products. In the proposed system, head orientation, body orientation, and arm action, the multiple cues are integrated for the customer behavior recognition. The proposed system discretizes the head and body orientation of customer into 8 directions to estimate whether the customer is looking or turning to the merchandise shelf. Semi-Supervised Learning method is applied to optimize the training dataset and to generate an accurate classifier. As for the arm action recognition, a novel combined hand feature (CHF), which includes hand trajectory, tracking status and the relative position between hand and shopping basket, is proposed to describe different arm actions. The CHF is classified by Dynamic Bayesian Network (DBN) into different arm actions. A series of experiments demonstrate the effectiveness of the proposed methods and the performance to the developed system.