Video Analytics for Customer Emotion and Satisfaction at Contact Centers

Due to the high levels of competition in a global market, companies have put more emphasis on building strong customer relationships and increasing customer satisfaction levels. With technological improvements in information and communication technologies, a highly anticipated key contributor to improve the customer experience and satisfaction in service episodes is through the application of video analytics, such as to evaluate the customer's emotions over the full service cycle. Currently, emotion recognition from video is a challenging research area. One of the most effective solutions to address this challenge is to utilize both the audio and visual components as two sources contained in the video data to make an overall assessment of the emotion. The combined use of audio and visual data sources presents additional challenges, such as determining the optimal data fusion technique prior to classification. In this paper, we propose an audio-visual emotion recognition system to detect the universal six emotions (happy, angry, sad, disgust, surprise, and fear) from video data. The detected customer emotions are then mapped and translated to give customer satisfaction scores. The proposed customer satisfaction video analytics system can operate over video conferencing or video chat. The effectiveness of our proposal is verified through numerical results.