Judging a video by its bitstream cover

Classifying videos into distinct categories, such as Sport and Music Video, is crucial for multimedia understanding and retrieval. Traditional methods require video decompression to extract pixel-level features like color, texture, and motion, thereby increasing computational and storage demands. We introduce a novel direction for video classification that does not rely on pixel domain information. Instead, we use the sequence of video frame sizes extracted from compressed bitstreams as input for a ResNet-based deep neural network, without the need for bitstream decoding or parsing. This approach leverages information captured by modern video compression algorithms, particularly advanced spatial and temporal prediction methods found in modern video coding standards such as H.264/AVC, H.265.HEVC and H.266/VVC.