MCMSys: Multimodal Data Closed-Loop Management System for Autonomous Driving

In order to safely and efficiently drive in various complex and diverse environments, autonomous driving requires the installation of multiple sensors to obtain information from the surrounding environment. Currently, most autonomous driving solutions are installed LiDAR, cameras and other hardware sensors to acquire multimodal data individually. The acquisition and application of multimodal data require consideration of data synchronization in temporal and spatial dimensions. In this paper, we design a multimodal data closed-loop management system for autonomous driving, which we named MCMSys, including multimodal data acquisition, multimodal data processing, and multimodal data management and application. We investigate how to install multiple sensors on the vehicle, actuate these sensors to acquire multimodal data, and temporally synchronize and spatially align the multimodal data. We also designed processing methods for the acquired multimodal data, including preprocessing and perception using advanced multimodal fusion algorithms. We investigated options for multimodal data management and application, and we created a vast data transmission scheme with high-speed data transmission capability and storage capacity to actualize the storage and application of large-scale multiple-sensor data.