3DSEAVNet: 3D-Squeeze-and-Excitation Networks for Audio-Visual Saliency Prediction

Video saliency prediction is an important task in the field of computer vision. Most of the existing video saliency prediction methods only focus on image information, and the audio information is often ignored. This leads to an incomplete perception mode, which makes it difficult to achieve optimal performance. SENet is an excellent attention mechanism-based network. It significantly enhances the performance of 2D convolutional networks. However, whether the 3D convolutional network can be applied to this attention mechanism network remains to be studied. In order to solve the above problems, we propose a saliency prediction network for audio-visual fusion to extract and predict various information in videos. At the same time, we improve the traditional SENet to make it applicable in 3D convolutional neural networks and discuss its role. Compared with the state-of-the-art methods, our model has strong competitiveness in multiple data sets.