Development of a 3D Environment Representation System for Vehicles Using Sensor Fusion Techniques

This research develops a 3D surrounding environment representation system for vehicles using sensor fusion techniques. The system integrates mmWave radar for distance measurement and ESP32-CAM cameras for object classification using the YOLO algorithm. The radar detects object distances within a 5 -meter radius, while YOLO classifies objects into categories such as pedestrians, cars, motorcycles, trucks, and buses. To overcome the radar's limitation of detecting only one object at a time, the calculation of object positions is added by utilizing the bounding box information obtained from the object detection process using the YOLO method. The four ESP32-CAM cameras continuously capture frames, which are processed by the YOLO algorithm for real-time object classification. Sensor fusion is performed in three stages: decision-level, data-level, and feature-level fusion. This fusion enables real-time calculation of object positions ($x, y, z$), which are visualized in a 3D environment with a $1: 100$ scale, providing a dynamic, animated view of the vehicle's surroundings. The system achieves over 95% object classification accuracy, enhancing spatial awareness and driving safety.