A 2-Stage Model for Vehicle Class and Orientation Detection with Photo-Realistic Image Generation

We aim to detect the class and orientation of a vehicle by training a model with synthetic data. However, the distribution of the classes in the training data is imbalanced, and the model trained on the synthetic image is difficult to predict in real-world images. We propose a two-stage detection model with photo-realistic image generation to tackle this issue. Our model mainly takes four steps to detect the class and orientation of the vehicle. (1) It builds a table containing the image, class, and location information of objects in the image, (2) transforms the synthetic images into real-world images style, and merges them into the meta table. (3) Classify vehicle class and orientation using images from the meta-table. (4) Finally, the vehicle class and orientation are detected by combining the pre-extracted location information and the predicted classes. We achieved 4^th place in IEEE BigData Challenge 2022 Vehicle class and Orientation Detection (VOD) with our approach. Our code and project material will be available at https://github.com/inu-RAISE/VOD_Challenge

I. Introduction

A large amount of data is required to train using a Transformer [1] or Convolution Neural Networks (CNN) [2] based model, which is in the spotlight in the field of computer vision. In addition, to apply this model to the real world using this model, it is necessary to collect and construct data from the real environment. However, this work has a challenge because it takes a lot of time and cost. To solve this challenge, we study using virtual synthetic data, which is relatively inexpensive, and then study to improve the model that can operate in a real-world driving environment.