Infrastructure-based Perception with Cameras and Radars for Cooperative Driving Scenarios

Roadside infrastructure has enjoyed widespread adoption for various tasks such as traffic surveillance, traffic monitoring, control of traffic flow, and prioritization of public transit and emergency vehicles. As automated driving functions and vehicle communications continue to be researched, cooperative and connected driving scenarios can now be realized. Cooperative driving, however, imposes stringent environmental perception and model requirements. In particular, road users, including pedestrians and cyclists, must be reliably detected and accurately localized. Furthermore, the perception framework must have low latency to provide up-to-date information. In this work, we present a refined, camera-based reference point detector design that does not rely on annotated infrastructure datasets and incorporates fusion with cost-effective radar sensor data to increase system reliability, if available. The reference point detector design is realized with box and instance segmentation object detector models to extract object ground points. In parallel, objects are extracted from radar target data through a clustering pipeline and fused with camera object detections. To demonstrate the real-world applicability of our approaches for cooperative driving scenarios, we provide an extensive evaluation of data from a real test site.