Large Field of View Cooperative Infrared and Visible Spectral Sensor for Visual Odometry

Thermal images produced by the long-wavelength infrared (LWIR) camera are robust and independent from environmental illumination change. They can help the standard visible light camera working under the complicated environmental condition. Breaking through the traditional stereo multi-spectral sensor consisting of a visible-light camera and a LWIR camera, a novel architecture of large field of view (FOV) cooperated infrared and visible spectral sensor for visual odometry is proposed. The novel sensor is equipped with two visible cameras, four infrared cameras covering 120 degrees FOV in horizontal under both bands. Distribution of cameras and related peripheral devices are specifically designed which makes the sensor's volume less than 100 cm (length) × 10 cm (height) × 10 cm (width). The sensor's cameras calibration, distortion correction and measurement principle are elaborated. Feature-based method for visible and multi-windowed optimization-based image alignment for infrared is designed for the visual odometer based on the different imaging mechanism and distribution of cameras in the sensor. The frames and estimated poses management from both bands are proposed. Moreover, all proposed methodologies can be implemented in the sensor's embedded processor. The electrical power consumption is only 12W. Experiments of the sensor's evaluation are performed, experimental results show that large FOV cooperated multi-spectral cameras can efficiently improve the robustness of visual odometry. The real-time performance of the sensor is higher than 10fps with disparity map construction under both bands.