Hardware and Software Co-Design Approach for Background Subtraction in High-Resolution Videos

Full High-Definition video has become the predominant resolution for surveillance cameras. However, the real-time background subtraction methods on Internet of Things (IoT) edge devices remain a challenge. In this paper, we propose a hardware and software co-design approach for high-resolution video background subtraction based on an improved Content Adaptive Resizing Framework (CARF) method. We retain the core of the CARF algorithm, which preserves object boundary information by coupling downsampling and upsampling layers. To reduce the algorithm’s demands on hardware resources, we have made specific modifications to CARF for the background subtraction task. These modifications include incorporating edge detection algorithms to retain boundary pixels and ignore non boundary redundant pixels, as well as fine-tuning the algorithm’s details. The modified algorithm performs nearly as well as the original one but requires fewer resources, making it more suitable for hardware implementation. The entire method is implemented on the Xilinx Zynq MPSoC, utilizing the programmable logic (PL) for hardware acceleration of this framework and the processing system (PS) for background subtraction. We validate the real-time performance of the proposed method on 1080p resolution datasets and analyze resource usage.