Deep Neural Network-Based Dynamical Object Recognition and Robust Multiobject Tracking Technique for Onboard Unmanned Aerial Vehicle’s Computer Vision-Based Systems

Computer vision-based systems seem highly perspective for semantic analysis of the dynamical objects. However, considering dynamical object recognition and tracking from the unmanned aerial vehicle (UAV) the task to design a robust model for data association is highly challenging due to additional issues, e.g., image degradation, nonfixed object camera distance and shooting focus, and real-time issues. Thus, we propose an accurate deep neural network-based dynamical object recognition and robust multiobject tracking technique based on bidirectional LSTM with the optimized motion and appearance gates as a multiobject tracking backbone, supported by an advanced single-shot detector network improved with residual prediction model and implemented a DenseNet network as well as a YOLOv4eff network as feature extraction. The technique has been trained on VisDrone 2022 and UAVDT datasets with the side-shoot dynamical objects at a height of up to 50 m. The performance analysis on the test stage performed on seven metrics demonstrate that the proposed technique surpasses, by accuracy and robustness ability, other state-of-the-art techniques based on two cumulative MOTA and MOTP, as well as MT and IDsw. In particular, we have dramatically decreased the number of IDsw which implies a better capability to handle several occlusions, which is a desirable property in real-time multiple object tracking. We have pointed out the sensitivity of the tracking performance of our technique on the number of utilizing different sequence lengths and have defined an optimum. Finally, the applicability and reliability of the proposed technique for onboard UAV computer-based systems have been discussed.