Game Engine Based Multi-View Video Dataset Synthesis for Pedestrian Detection and Tracking

Multi-view deep learning models have demonstrated significant promise in addressing pedestrian detection and tracking challenges, such as heavy occlusion in monocular cameras and their restricted field of view. However, these models demand a considerable volume of training data, the acquisition of which is time-consuming, labour-intensive, and further complicated by privacy and ethical concerns. The currently available public multi-view datasets are insufficient to support the extensive training required for these models.To solve the paucity of multi-view training data, this paper presents a novel multi-view synthetic dataset pipeline, named WildPerception, based on integrated techniques of Unity Perception, SyntheticHumans Package and MultiviewX. WildPerception simulates pedestrians in a photo-realistic scene along with multiple overlapping views, allowing an instant generation of large-scale and labeled video training datasets in WILDTRACK format. Our pipeline is modular and can be easily tailored to the demands of diverse specific multi-view tasks. Experiments were carried out to validate the efficiency of this pipeline.Moreover, the models trained on these synthesized datasets also benefit the robust adaptability when deployed on datasets gathered from novel environments.The code for the pipeline is publicly available on GitHub at https://github.com/TsingLoo/com.tsingloo.wildperception to facilitate reproducibility and further research.