Toward Real-World Applicability: Lightweight Underwater Acoustic Localization Model Through Knowledge Distillation

Deep learning (DL) approaches in underwater acoustic localization (UAL) have gained a great deal of popularity. While numerous works are devoted to improving the localization precision, they neglect another critical challenge inherent in the DL-based UAL problem, i.e., the model's practicality. Advanced DL models generally exhibit extremely high complexity, requiring a large amount of computational resources and resulting in slow inference time. Unfortunately, the limited processing power and real-time demands in oceanic applications make the deployment of complex DL models exceedingly challenging. To address this challenge, this article proposes a lightweight UAL framework based on knowledge distillation (KD) techniques, which effectively reduces the size of a deep UAL model while maintaining competitive performance. Specifically, a dedicated teacher network is designed using attention mechanisms and convolutional neural networks (CNNs). Then, the KD is performed to distill the knowledge from the teacher network into a lightweight student model, such as a three-layer CNN. In practical deployment, only the lightweight student model will be utilized. With the proposed lightweight framework, the student model has 98.68% fewer model parameters and is 87.4% faster in inference time compared to the teacher network, while the prediction accuracy drops to only 1.07% (97.55% $\rightarrow$ 96.48%). In addition, the generalization ability of the student model is examined through transfer learning, where the model is transferred between two different ocean environments. The student model demonstrates a stronger generalization ability compared to the model without the KD process, as it can quickly adapt itself to a new application environment using just 10% of the data.