Complexity-Driven Model Compression for Resource-constrained Deep Learning on Edge

Recent advances in Artificial Intelligence (AI) on the Internet of Things (IoT) devices have realized Edge AI in several applications by enabling low latency and energy efficiency. However, deploying state-of-the-art Convolutional Neural Networks (CNNs) such as VGG-16 and ResNets on resource-constrained edge devices is practically infeasible due to their extensive parameter counts and floating-point operations (FLOPs). Thus, the concept of network pruning as a type of model compression is gaining attention for accelerating CNNs on low-power devices. State-of-the-art pruning approaches, either
structured
or
unstructured
, postulate a three-stage:
training-pruning-retraining
pipeline, which results in an inevitable retraining overhead. In this work, we posit an orthogonal conjecture on structured pruning at
initialization
to find sparse subnetworks realizing ≈ 60% less training time than conventional pruning. Moreover, conventional pruning focuses on identifying the saliency at filter granularity while ignoring the importance of characteristics at layer granularity. In contrast, the proposed complexity-driven approach leverages the intrinsic complexities of CNN layers to guide the filter-pruning process without requiring dense pre-training of models. Particularly, we characterize the importance of CNN layers with respect to parameters, FLOPs, and memory-based complexities to work in tandem with filter-pruning in a structured manner. Experiments show the competitive performance of our approach in terms of accuracy and acceleration for all three modes of pruning, namely parameter-aware (PA), FLOPs-aware (FA), and memory-aware (MA). For example, reducing ≈ 70% parameters, ≈ 50% FLOPs, and ≈ 50% memory from MobileNetV2 did not result in any accuracy loss, unlike state-of-the-art approaches. Lastly, we present a trade-off between different resources and accuracy, which can be helpful for developers in making the right decisions in resource-constrained IoT environments.