Panoramic virtual stereo vision of cooperative mobile robots for localizing 3D moving objects

Flexible, reconfigurable vision systems can provide an extremely rich sensing modality for sophisticated multiple robot platforms. We propose a cooperative and adaptive approach of panoramic vision to the problem of finding and protecting humans by a robot team in an emergency circumstance (e.g., a rescue in an office building). A panoramic virtual stereo vision method is proposed for this cooperative approach, which features omni-directional visual sensors, cooperative mobile platforms, selected 3D matching, and real-time moving object (people) detection and tracking. The problems of dynamic self-calibration and robust 3D estimation of moving objects are discussed. A careful error analysis of the panoramic stereo triangulation is presented in order to derive rules for optimal view planning. Experimental results are given for detecting and localizing multiple moving objects using two cooperative robot platforms.