Convolutional Neural Networks for Image Emotion Recognition by fusing Differential and Supplementary Information

Emotions arise out of complex phenomena, which are believed to have a biological basis. Neuroscience research has demonstrated that emotions are related to distinct patterns of brain activity and the release of certain chemicals, such as hormones and neurotransmitters, into the bloodstream. Emotion recognition is used in many applications, such as advertising, social networking, and cinema. In this paper, we propose a deep convolutional neural network (CNN) fusion technique made up of two parts: a differential-CNN system to extract emotional features from an image and combine them using convex combination and a supplementary CNN to extract central object details of an image and combine it with differential-CNN features. We amplify the difference between minute latent representations of an image via differential-CNN features. The type of emotion an image elicits is highly influenced by its central object. Hence supplementary CNN helps supplement the differential-CNN features to improve emotion recognition. In comparison to the contemporary state-of-the-art methods, our proposed method showed an absolute gain of 6% on the primary dataset and outperformed them on the different secondary datasets social media web scraping process.