Integrated Deep Learning System for Car Damage Detection and Classification Using Deep Transfer Learning

The process for claiming car insurance in Indonesia often takes a long time. Therefore this research aims to speed up the process of car insurance claims by making an integrated deep-learning system. Three main tasks can segment car damage, detect damaged parts, and classify the severity of car damage. The dataset consists of 3 parts; the damaged segmentation dataset, the car parts segmentation dataset, and the car damage classification dataset. The models used in this paper are trained using deep transfer learning methods to save time and resources. The model for the segmentation task is Mask R-CNN, while for the classification task, we compare EfficientNet and MobileNetV2. In this paper, we try to increase the F1 score by adding a concatenate simple CNN and using the output from damage segmentation as an additional input for the classification model. Experiments show that Mask R-CNN is capable of detecting damaged car parts. Concatenate simple CNN modification on the classification model can boost the model up to a 9% average difference on the F1 score. The best classification model is MobileNetV2 which uses simple CNN modification with an F1 score of up to 91%.