GIBS-Net: Unseen Video Background Subtraction with Global Information

Background subtraction (BGS) is a fundamental task in video processing and serves as the basis for many downstream applications in computer vision. While existing deep learning-based supervised algorithms have achieved remarkable results on certain datasets, their performance may deteriorate when applied to unseen videos. The current BGS algorithms for unseen video primarily focus on utilizing local information from video frames, resulting in limited access to global information. However, this limitation affects the overall performance of the model in accurately separating the foreground objects from the background. Consequently, we sought to address this limitation by incorporating global video information into the model verification process to enhance detection performance. In this study, we propose an improved model, GIBS-Net, by leveraging the Version Transformer (ViT) to enhance spatiotemporal connectivity of data. Our experiments demonstrate that GIBS-Net achieves superior testing performance on the CDNet-2014 dataset compared to state-of-the-art algorithms evaluated on unseen videos in terms of various metrics such as F-measure, recall, and precision. Furthermore, GIBS-Net exhibits excellent performance in processing unseen videos under abnormal weather conditions. In addition, we have conducted lightweight processing on the model and proposed the GIBS-Net-S model, which significantly reduces the number of parameters and floating point operations per second (FLOPs) while maintaining a high frames-per-second (FPS) rate.