Convolutional Transformer for Fast and Accurate Gravitational Wave Detection

Recent studies utilize convolutional neural networks (CNNs) to detect gravitational waves (GWs) from cosmic noises. However, the performance of these models is limited by the inherent local receptive field of CNNs. To address this issue, we propose a new GW detection architecture, named convolutional transformer (CoT), that integrates self-attention and convolution mechanisms elegantly. CoT employs self-attention to capture long-term dependencies via a global receptive field. Meanwhile, CoT leverages convolutional projection to reduce computation cost and complement inductive bias of locality. The hybrid architecture allows us to aggregate informative features and detect GW signals more effectively and efficiently than purely convolution-based architectures. Moreover, we integrate CoT with spectral self-attention that attends to both the time and frequency axes, which better exploits frequency features in the spectrogram representation. Experimental results on the LIGO dataset demonstrate that CoT is superior to CNN-based methods in accuracy (0.6% higher AUC/F1 than 2D-CNN) and inference speed (1.15 times speedup than 1DCNN). To the best of our knowledge, this is the first time the self-attention mechanism has been employed in GW detection.