Gated Mechanism Attention Transformer Based on Wavelet Enhanced Optical Flow Field Estimation for Foreground Detection

Detecting foreground objects in video separation tasks is a challenging endeavor in complex environments. This paper presents a deep neural network architecture considering the features of spectral, spatial, and temporal at the same time. Under this framework, we designed a gated mechanism for attention and incorporated it into the Transformer architecture (GMAT). This model employs a gating mechanism to dynamically control and allocate attention across different features (wavelet features and raw features), learning how to balance their importance, emphasize or ignore certain features based on the current context. Additionally, in order to enhance features in video frame data and better focus on important features while ignoring unimportant ones in GMAT, we introduced an optical flow estimation method based on wavelet transform. Due to the advantage of wavelet transform in capturing motion features at different scales, its introduction enables a more comprehensive and detailed focus on finer features. We evaluated the GMAT on various videos from the CDNet2014 dataset, results of qualitative and quantitative evaluations demonstrating its significant improved for foreground detection over several recent video separation models.