SVT: Spectral Video Transformer for Video Restoration in Under-Display Camera

Under-display camera (UDC) systems enable full-screen view experience on smartphones. However, the pixel array of the semi-transparent Organic Light-Emitting Diode (OLED) in the display diffracts and attenuates the incident light, which introduces multiple degradations in captured videos. The degradations in UDC videos vary both spatially and temporally, which brings challenges for UDC video restoration. In this paper, we present a novel transformer-based method called Spectral Video Transformer (SVT) for UDC video restoration. For spatial modeling, SVT employs SpecFormer, which leverages spectral transform to balance global and local restoration simultaneously. For temporal modeling, a Local-Aware Temporal Transformer integrates features within local windows across neighboring frames using temporal cross-attention. To illustrate the superiority of SVT, we build a large-scale UDC video dataset based on a two-stage model-based UDC synthesis pipeline. Extensive experiments demonstrate the superiority of SVT over other models quantitatively and qualitatively.