A Versatile Application for Visual Simultaneous Localization and Mapping with Object Detection

This paper proposed a novel system with visual SLAM and an embedded object detection to achieve real-time object detection and location tracking. OpenVSLAM was used as it incorporates several useful features and functions for research and development. Parallelly YOLOv5, an AI-based object detection model was chosen due to its lightweight model being able to work at very fast speeds. As both OpenVSLAM and Yolov5 could be run in real-time, the proposed system essentially runs both models in parallel to make it a real-time application. In our system, the OpenVSLAM API is used to build 3D maps and determine the pose of the camera in a Linux based operating environment. This is to simulate the digital map based on the image/frame that was detected from a given camera. The Yolov5 module works on pre-trained model by integrating with OpenVSLAM to make the object detection and VSLAM mapping at minimum latency. As Yolov5 is readily available to be implemented in python, the python program was developed working under a virtual environment created with the usage of Anaconda. To provide the users with the best experience, the final product of this paper includes a GUI (Graphical User Interface) to work alongside the main program-flow. Experimental results showed that the proposed system is effective in both detect the object and show the location of the object in the real-time generated map. The system can be used in many applications like discovering trapped individuals, firefighting, rescuing missions.