Classification of facial expressions using self-organizing maps

Just as humans use body language or nonverbal language such as gestures and facial expressions in communication, computers will also be able to communicate with humans. In medical engineering, it is possible that recognition of facial expression can be applied to support communication with persons who have trouble communicating verbally such as infants and mental patients. The purpose of this study is to enable recognition of human emotions by facial expressions. Our observations of facial expressions found that recognizing facial expressions by identifying changes in important facial segments such as the eyebrow, the eyes and the mouth by using sequences of images is important. Self-organizing maps, which are neural networks, are used to extract features of image sequences. The image sequences of six types of facial expressions are recorded on VTR and made into image sequences consisting of 30 images per second. Gray levels of each segment are input into the self-organizing map corresponding to each segment. The neuron in the output layer, called the victory neuron, reacts to the feature nearest the input segment. Our analysis of the changes in victory neurons demonstrates that they have characteristic features which correspond to each of the six facial expressions.