Novel Dilated Separable Convolution Networks for Efficient Video Salient Object Detection in the Wild

Appearance and motion are essential features in video salient object detection (VSOD) tasks. Most of the existing approaches use local features and thus fail to understand both the appearance and motion-specific semantics at the global level. Hence, these methods are unable to perform in unconstrained scenarios where multiple challenges, such as partial occlusion, motion blur, noise, and clutter background, exist. Moreover, these approaches require a large number of computational resources due to their complex structures, which limits their applicability to real-world deployment. To resolve these issues and to achieve a balance between accuracy and computational complexity, in this article, a dilation separable convolution network (DSCNet) is proposed, which is equipped with dilation attention fusion module (DAFM), bidirectional cross-modality fusion module (BCFM), and saliency prediction module (SPM) to extract enhanced multiscaled motion and appearance features without increasing the model complexity. Furthermore, a bidirectional separable convolution network (BSCNet) equipped with a separable convolution module (SCM) and a FlowNet2.0 is proposed to use multiscale contextual information across appearance cues and generate enhanced multiscaled motion maps. For faster and better training of the DSCNet model, we propose a novel stochastic-gradient-based firefly algorithm (SGFA), which adaptively balances the exploration and exploitation in multiscaled, cross-modal embedded subspaces. With the help of the proposed SGFA algorithm, the DSCNet+ model is constructed on top of DSCNet, which further improves the results in terms of the training speed and other evaluation metrics. The proposed models are evaluated on six benchmark datasets, and a detailed comparative study is provided with 16 state-of-the-art (SOTA) models. One of the major highlights of the work is the significant performance of the proposed models on the most difficult DAVSOD-Diff dataset, which best reflects the challenging real-world scenarios.