Social Signal Processing [Exploratory DSP]

Face-to-face communication conveys social context as well as words. It is this social signaling that allows new information to be smoothly integrated into a shared, group-wide understanding. Social signaling includes signals of interest, determination, friendliness, boredom, and other "attitudes" toward a social situation. Psychologists speculate that social signaling may have evolved as a way to establish hierarchy and group cohesion because social signaling functions as a subconscious discussion about relationships, resources, risks, and rewards. In many situations the nonlinguistic signals that serve as the basis for this social discussion are just as important as conscious content for determining human behavior. In what follows we discuss challenges in exploratory processing of social signals and tools that allow us to predict human behavior and sometimes exceed even expert human capabilities. These tools potentially permit computer and communications systems to support social and organizational roles instead of viewing the individual as an isolated entity. Example applications include automatically patching people into socially important conversations, instigating conversations among people in order to build a more solid social network, and reinforcing family ties.