Structure-aware Roadview Synthesis for Testing Autonomous Perception

To understanding the generalization of visual perception for autonomous vehicles, sufficient and diverse enough test samples need to be extracted from traffic scenarios. However, the “long-tailed” effect makes it exhausted and expensive to acquire data from tail categories e.g. scenarios backboned with S-shape winding roads. Instead of rendering semi-realistic images in graphics engines like Unity 3D, we propose to synthesis road views with given projection of any road structure by exploiting the statistical relationship between virtual-real image pairs. To tackle non-uniform structural misalignments in the learning data, we introduce a latent structure similarity constraint in the genre of cycle consistency. In this case, synthesized views can better retain the required road's geometry while being rendered with realism. Experimental results demonstrate the effectiveness of the proposed method even when large and massive misalignments exist in training pairs. Further study reveals the direct or indirect impact on autonomous perception caused by road's varying geometry.