Anomaly detection in complex data: a practical application when outliers are few

Ocean Networks Canada (ONC) runs a cabled ocean observatory that, among a wide variety of oceanographic instruments, has more than 30 hydrophones deployed. These hydrophones continuously send audio data back from the ocean floor, resulting in greater than 7 TB of data per hydrophone per year. To quickly visualize these acoustic data, power spectral density plots (spectrograms) are generated and stored in the ONC archive. While hydrophones generally perform well in the field, losses in sensitivity or ‘dropouts’ occasionally occur at isolated frequencies or across all bands for short periods of time. While such dropouts can be spotted on a spectrogram via visual inspection, the manual review of thousands of images is costly and impractical. With the current number of ONC’s hydrophones, it takes a data specialist over eight hours to inspect a day’s worth of spectrograms. We used a total of 2185 spectrogram images to train an autoencoder model. These images were split into three data sets: regular spectrograms without anomalies (1024 images that were separated in training and validation data sets), test data set (1000 unclassified images) and spectrograms containing anomalous signals (161 images). We used mean squared error (MSE) as a metric for the image reconstructions provided by the model and to establish the threshold of 0.006 to identify anomalous spectrograms. The trained model was capable of identifying most anomalous images from regular ones and correctly labelled approximately 93% of the images with anomalous features in the anomalous data sets.