Video Description Method Based on Contextual Memory Network and Gated Fusion

The deficiency of object representation in the model can result from the absence of contextual information in video captioning tasks. In addition, taking into account the intricate correlation between visual and semantic features, adequately exploiting both types of features often presents a challenging task. To tackle the aforementioned issues, this paper proposes a video description method based on contextual memory network and gated fusion. Firstly, a contextual memory network is proposed to store contextual information from previous video frames for more accurate object information capturing. Secondly, a feature fusion network based on gating mechanism is designed, which can adaptively aggregate visual and semantic features, resulting in enhanced richness and effectiveness of feature descriptions. Experimental results demonstrate that the proposed method, tested on the MSVD dataset, can generate descriptions that are more specific and complete, and achieve better results than the existing state-of-the-art methods in various metrics.