Towards an intelligent robotic walker for assisted living using multimodal sensorial data

We aim at developing an intelligent robotic platform that provides cognitive assistance and natural support in indoor environments to the elderly society and to individuals with moderate to mild walking impairment. Towards this end, we process data from audiovisual sensors and laser range scanners, acquired in experiments with patients in real life scenarios. We present the main concepts of an automatic system for user intent and action recognition that will integrate multiple modalities. We demonstrate promising preliminary results, firstly on action recognition based on the visual modality, i.e. color and depth cues, and secondly on the detection of gait cycle patterns that exploit the laser range data. For action recognition we are based on local interest points, 3D Gabor filters and dominant energy analysis, feeding a support vector machine. Then the recognized actions can trigger the gait cycle detection that detect walking patterns by exploiting the laser range data, modeled by hidden Markov models. In this way, we shall acquire the overall patient's state and the robot shall autonomously reason on how to provide support.