Agnostic Feature Compression with Semantic Guided Channel Importance Analysis

Distributing the computational workload of neural networks across cloud servers and local devices is an effective strategy for deploying resource-intensive deep models to edge devices. Therefore, compressing the deep features without compromising performance is crucial for saving server storage and transmission bandwidth. However, existing feature compression approaches are model or task specific and require training from scratch. In this paper, we propose a general and efficient framework for compressing deep features without requiring any prior knowledge of the semantics or task of the features. Our key observation is that different parts of the the feature map have different importance levels for a specific task. We can apply compression operation to a deeper degree for less irrelevant parts to achieve a high compression rate, while preserving the performance by applying a lower compression ratio to the more important parts. Focusing on this idea, we use the activation map generated by GradCAM [1] to classify each deep feature channel into essential and peripheral categories. To improve classification accuracy, we utilise semantic segmentations to provide natural boundaries for scoring each semantic channel. Peripheral channels are compressed using binary compression to achieve a high compression rate, while essential channels are compressed using mask compression. To effectively separate the essential and peripheral channels for a given input feature, we adopt a data-driven approach to identify the essential channels from datasets. Experimental results demonstrate that our method outperforms the state-of-the-art feature compression methods and is generalizable for various deep models.