Approach to semi-automatic labeling of video sequences for affective computing-enabling the comprehensive assessment of emotion detection software from mimics

In recent years, most breakthroughs in fields such as image and video processing were based on machine learning technologies that allow computers to recognize objects in images with nearly human precision. In some application domains, computers even surpassed human level performance. These breakthroughs result from an exponential increase of computational resources and digitization of society (massive availability of video material) as well as significant progress in the field of artificial intelligence, namely deep learning. Despite these advances, however, automatically recognizing emotions of humans in video material is still a challenging, but important task in affective computing. Without doubt, the automated detection of social-emotional and psychiatrically relevant information from video material will make a valuable contribution to various domains. This is particularly true for the field of personalized medicine. Major obstacles are high efforts and costs of labeling single video frames manually to create training data for machine learning technologies. To tackle this, we outline and discuss a methodological approach to create labeled video data with a minimum of human involvement.