Optimal Deblocking Filtering for H.264 Intra-Prediction Mode Decision

Video coding standard H.264 exhibits blocking artifacts in its conventional operations due to the inherent characteristics of block-based discrete cosine transforms. A significant amount of research has been done over many years to reduce the blocking artifacts in compressed videos. Therefore, deblocking filters are highly recommended for minimizing blocking artifacts in compressed reconstructed video sequences. Additionally, it improves the signal’s visual quality and prediction performance. The prediction performance gain could also be achieved by smoothing the sharp edges between macro-blocks. However, the in-loop structure of deblocking filter design lacks computational efficiency and also does not ensure the detection of true edges from the image’s subjective and objective perspectives. This study presents a novel framework for optimizing H.264 integrated deblocking filters. Customized filtering ensures that the enhanced signal quality can be reconstructed by discriminating between true edges and artifacts during the intra-frame prediction process. The experiments demonstrate that it can achieve better signal quality while reducing encoding time and gaining coding performance.