A Simple, Efficient and Complete Software Pipeline for Constructing an RGB-D Camera Using RGB and ToF Depth Cameras

Robotics and computer vision applications have utilized consumer-grade RGB-D cameras because of their low cost and usability for various applications. These cameras capture RGB images with per-pixel depth information. The construction of RGB-D cameras using RGB and ToF depth sensors introduces significant challenges due to differences in resolutions and fields of view (FOV), leading to misalignments between RGB and depth data streams. This work proposes a simple, efficient and complete pipeline for constructing an RGB-D camera using RGB and ToF depth sensors, addressing the challenges of aligning data streams with differing resolutions and fields of view (FoV). The effectiveness of the pipeline is validated using experimental comparisons with existing RGB-D camera systems, including Intel RealSense and Vzense cameras. The registered depth and point clouds generated by the proposed approach were benchmarked against these systems using various datasets. Different performance metrics have been used to assess the effectiveness of these cameras, such as a number of holes in the registered depth image, object boundaries, color mixing at object edges and alignment of RGB and depth images. The performance of these cameras was evaluated under no-light conditions. Furthermore, quantitative analysis has been performed in terms of missing pixel (%) and repeatability to evaluate the performance of all cameras. The results demonstrate that the proposed pipeline achieves good alignment accuracy, improves depth data usability, and offers a practical solution for applications like 3D reconstruction and object detection.