An HMM-based temporal difference learning with model-updating capability for visual tracking of human communicational behaviors

In the development of natural interaction for systems, the trend of using vision toward translating human actions into instruction symbols is arising for the recognition of non-verbal communication channels. We propose an adaptive vision-based attentive tracker (AVAT) to track human intended communicational actions with attentive zooming capability as well as an exploration function for model updating. The AVAT isolates such human intended actions from the ordinary walking behavior based on an algorithm with two subprocesses: one is for modeling the movement of human body parts as the environment using HMMs (Hidden Markov Models), and the other is for learning the model of the tracker's action using a model-based TD (Temporal Difference) algorithm. We describe the integration of the two algorithms and then derive the model updating formulas from the newly optimized TD policies. An experimental result of isolating the human sign action during his natural walking motion is shown for demonstrating the feasibility of our system. Identification of the sign gesture context using a confirmation method using wavelet analysis provides rewards for optimizing the tracker's action models.