Object Detection and Classification Based on their 3D Models

This paper proposes a new approach to detect and classify terrestrial and aquatic objects based on their 3D models. The advantage of this approach is its independence from the number of output classes, which allows to dynamically add or remove them, as well as no need for a training sample with images of real objects. We propose two-stage image processing: 1) preliminary object detection and segmentation using YOLOv8, 2) transformation of the segmented object into an embedding vector for matching with vectors in the available database based on 3D models. The second step allows to match the nearest 3D object angle to the detected object and unambiguously classify it. The conversion of the segmented object into an embedding vector is performed using a neural network trained based on triplet loss. The approach was tested on a set of 3D ship models and their real videos captured from drones and other locations. The results showed an accuracy of 95% on the test sample when classifying a given set of objects.