People tracking using integrated sensors for human robot interaction

In human-human interaction, position and orientation of participants' bodies and faces play an important role. Thus, robots need to be able to detect and track human bodies and faces, and obtain human positions and orientations to achieve effective human-robot interaction. It is difficult, however, to robustly obtain such information from video cameras alone in complex environments. Hence, we propose to use integrated sensors that are composed of a laser range sensor and an omni-directional camera. A Rao-Blackwellized particle filter framework is employed to track the position and orientation of both bodies and heads of people based on the distance data and panorama images captured from the laser range sensor and the omni-directional camera. In addition to the tracking techniques, we present two applications of our integrated sensor system. One is a robotic wheelchair moving with a caregiver; the sensor system detects and tracks the caregiver and the wheelchair moves with the caregiver based on the tracking results. The other is a museum guide robot that explains exhibits to multiple visitors; the position and orientation data of visitors' bodies and faces enable the robot to distribute its gaze to each of multiple visitors to keep their attention while talking.