Utilizing Computer Vision Algorithms to Detect and Describe Local Features in Images for Emotion Recognition from Speech

One of the core problems of machine learning applications, and in turn when recognizing Emotions from speech, is the difficulty to decide which measurable features are the ones containing the relevant information concerning the emotion classification task. As there is a wide variety of feature sets extractable from audio signals, which all have different origins as well as advantages and disadvantages, one should choose a method that provides an easy search for relevant features and helps with the selection process. The novelty in our contribution is using methods concentrating on the visual distinction of spectrograms generated from emotionally loaded utterances. The aim was the improvement of the search for strongly emotion-dependent areas in spectrograms, which can be used for easy and efficient emotion classification tasks. For this we employed methods proven to work for similar problems with spectrograms. In this research, the Oriented FAST Rotated BRIEF (ORB) was selected as feature extraction algorithm, a method which is based on the Binary Robust Independent Elementary Features (BRIEF) extraction. The local features were computed from the Smartkom database, which were translated from audio recordings into visual spectrogram representations. Afterwards a Support Vector Machine (SVM) classifier was trained to recognize the emotions for a seven-case (Emotion Classes) and two-case (Valence/Arousal Distinction) problem. This proposed method attained high recall scores on this specific database comparable or in excess of similar methods in the literature. Different parameter settings, like window length, step size in spectrogram creation, denoising of spectrograms and the number of keypoints per spectrogram were analyzed to validate the impact on the classification performance.