The classification performance of signal-to-noise ratio and kinematic features in varying environments

The use of high resolution, broadband sonars during anti-submarine warfare operations often result in a sonar image cluttered with false alarms. This may prove challenging for a sonar operator when it comes to correctly distinguishing sonar contacts from false alarms. These false alarms may originate from different sources such as seamounts, underwater ridges, topographical features in general or man-made objects, such as pipelines. The unmanagable amounts of false alarms are today handled most commonly by raising the detection threshold on the signal-to-noise ratio. This would, however, also lower the probability of detecting a valid target, such as a submarine. At track level, false alarms from topographical features, such as seamounts, should be very different from those resulting from pipelines, hence, leading to differences in their kinematic properties. Machine learning should then be able to exploit these different sources of information in order to separate actual submarine detections from false alarms. However, it is well known that the application of such methods on a dataset from one environment is not necessarily valid for a different environment. Here we apply the above mentioned method on two fundamentally different datasets. The first dataset, Clutter Experiment 2 (CEX02), was collected close to the coast where the false alarms mainly consist of returns from a rocky, upslope bottom. The second dataset, Localization Experiment 2 (LOC02), is from the middle of the Norwegian trench, an area characterised by a very flat bottom and far fewer false alarms. Both datasets were collected in 2002 during the New Array Technology III (NAT III) programme, which was a combined effort between the Norwegian Defence Research Establishment (FFI), Thales Underwater Systems, TNO, the Royal Norwegian Navy, the Royal Dutch Navy, and the French Navy. The trial was carried out in the Norwegian Trench where a low-frequency source and an array receiver were towed by FFIs research vessel, HU Sverdrup II. The data are processed up to track level before the kinematic properties and signal-to-noise ratio of both false tracks and target tracks are extracted. Due to the confidential nature of submarine tracks these are removed from the analysis and pipeline tracks are considered as target tracks. An important find was that automatic classifiers trained through machine learning on data from one environment does not necessarily work in a different environment. Furthermore, we observe that classifiers tested on data from the same location as the training dataset exhibit signs of overtraining. Finally, the best performance in all instances is found when the classifiers are determined from a combined dataset of both environments before applied to the different environments separately.