Removing Bias of Video Question Answering by Causal Theory

Large-scale pre-trained models and training datasets are crucial for bridging the gap between language visual modalities and enhancing the reasoning ability of the video question-answering model. However, the pre-training data often contains significant harmful bias, and the training data exhibits long-tailed distributions of attributes. This results in a model trained with bias, which contradicts our need for unbiased inference. In this work, we propose a method rooted in causal theory to eliminate bias in VideoQA. We first perform a causal analysis on the video question-answering task to understand how prior bias impacts prediction. We then apply counterfactual reasoning to exclude the bias induced by language and visual priors separately. Furthermore, to mitigate the influence of imbalanced attributes in the training data, we construct a new resampling environment to ensure that feature centers remain stable in all environments. According to the experimental results, our model achieves competitive performance on the MSVD-QA dataset and the NExT-QA dataset.