BHSE-VQA: A Bidirectional Hierarchical Semantic Extraction Structure for Video Quality Assessment

The diversity of video content and unpredictability of distortions in user-generated content (UGC) videos pose a challenge for video quality assessment (VQA). Most existing methods are difficult to model complete visual perception loop to accurately capture video content and predict perceived quality. Thus, as shown in Figure 1 , this paper proposes a bidirectional hierarchical semantic extraction structure for VQA (BHSE-VQA), which simulates visual feedforward and feedback perception processes. Firstly, the feedforward and feedback multi-level network (FFMNet) based on the reverse hierarchy theory is designed to extract and adjust hierarchical semantic features on the bidirectional pathway. Then, considering the different effects of feature depth on visual perception results, this paper suggests a multi-level weight redistribution (MWR) strategy that makes use of the attention characteristics of the feedback outputs to realign the weights at each stage of the feedforward outputs. Finally, through a temporal attention fusion network (TAFNet), this paper further extracts the detailed features arising from feedforward and feedback perceptual differences and obtains quality scores. The experimental results show that the proposed model exceeds 0.845 on both SRCC and PLCC on YouTube-UGC database.