SET: Stereo Evaluation Toolbox for Combined Performance Assessment of Camera Systems, 3D Reconstruction and Visual SLAM

Evaluating stereo reconstruction algorithms regardless of camera system and application environment is not sufficient to rate the overall performance of a stereo system. To overcome this the Stereo Evaluation Toolbox (SET) proposes a well-founded selection and comparison approach for stereo systems. We aim at providing one performance score evaluating the generated stereo point cloud, complementary to common benchmarks which solely evaluate stereo algorithms on provided image sets. Using images captured in the desired application environment SET measures and compares performance as interaction of modular camera-algorithm combinations inside a local application scenario. Furthermore an evaluation approach for camera-based visual simultaneous localization and mapping (SLAM) systems is presented for further analysis. We apply SET on the evaluation of different camera systems in 3D reconstruction of indoor and outdoor environments as well as on visual SLAM for person indoor navigation.