More than Vision: A Multi-Modal Assisted Model to Text-Video Retrieval

The rapid growth of video on the internet has made searching for video content using natural language queries a significant challenge. Text-video retrieval is a challenging task that aims to search relevant video content based on natural language descriptions. However, existing methods for caption-to-video retrieval primarily focus on the visual representation of videos and do not fully exploit the rich potential of multi-modal information inherently present in videos. In this paper, we propose a novel multi-modal approach: more than vision MTV that leverages the diverse cues available in videos. Our approach aims to bridge the gap between textual descriptions and visual content by incorporating multi-modal information. By integrating action, face, and audio multi-modal cues, it can better help the core visual cues to fully understand the video content, leading to enhanced retrieval accuracy and performance. Through extensive experiments and evaluations on standard benchmarks, we demonstrate the effectiveness and superiority of our multi-modal approach in text-video retrieval tasks.