Combined 2-D sound source localization with stereo vision for intelligent Human-Robot Interaction of service robot

We have developed an intelligent service robot which consists of several sensors and functions, such as stereo vision, sound source localization, remote supervisory and locomotion. The objective of this paper is to address a 3-dimensional sound source localization method in localizing the position of user while the user gives a command. To achieve this goal, stereo vision and audition system is combined to acquire speaker's position in 3-D coordinate. The depth information could be obtained by stereo vision, and the direction information could be obtained by audition system. We place coplanar microphone array and two identical cameras on the robot to implement this idea. In addition, an agent system is designed to combine the speech recognition system with several services which can provide more information to users and communicate with other devices.