FPX-NVC: An FPGA-Accelerated P-frame Based Neural Video Coding System

Hardware-oriented neural video coding (NVC) has been an emerging research topic recently. In this work, we present a P-frame based NVC system by leveraging the parallel neural inference capability of resource-constrained embed platforms, such as field programmable gate array (FPGA). The proposed system, namely FPX-NVC, consists of multiple resolution video capture, reconfigurable and integerized neural video codec and display. Our neural codec enjoys the learned unidirectional inter-predictive coding by jointly optimizing the implicit motion representation (IMR), feature domain motion compensation, and the corresponding prediction error compression, realizing end-to-end on-board encoding and decoding. Our system supports end-to-end video data operation flow from capturing to neural coding and display. To our knowledge, the FPX-NVC is the first inter-predictive coding (P-frame) based hardware NVC system.