The TALA Empathic Space: Integrating Affect and Activity Recognition into a Smart Space

Advancement in ambient intelligence is driving the trend towards innovative interaction with computing systems. In this paper, we present our efforts towards the development of the ambient intelligent space TALA, which has the concept of empathy in cognitive science as its architecture's backbone to guide its human-system interactions. We envision TALA to be capable of automatically identifying its occupant, modeling his/her affective states and activities, and providing empathic responses via changes in ambient settings. We present here the empirical results and analyses we obtained for the first two of this three-fold capability. We constructed face and voice datasets for identity and affect recognition and an activity dataset. Using a multimodal approach, specifically, applying a decision level fusion of independent face and voice models, we obtained accuracies of 88% and 79% for identity and affect recognition, respectively. For activity recognition, classification is 80% accurate even without employing any fusion technique.