DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Dataset

High data-rate and low-latency vehicle-to-vehicle (V2V) communication is essential for future intelligent transport systems to enable coordination, enhance safety, and support distributed computing and intelligence requirements. Developing effective communication strategies, however, demands realistic test scenarios and datasets. This is important at the high frequency bands where more spectrum is available. However, higher frequency bands require directional transmission and are sensitive to propagation blockages. To enable the study of such challenges, this work presents the first large-scale multi-modal dataset for mmWave vehicle-to-vehicle communications. It presents a two-vehicle testbed that comprises data from a 360$^{\circ }$ camera, four radars, four 60 GHz phased arrays, a 3D lidar, and two precise GPSs. The dataset contains vehicles driving during the day and night for 120 km in intercity and rural settings, with speeds up to 100 km per hour. More than one million objects were detected across all images, from trucks to bicycles. This work further includes detailed dataset statistics of various real-world scenarios and highlights how this dataset can enable novel machine-learning applications.