Improving INT Inference Resilience of CNNs by Vulnerability-Aware Mixed-Precision Quantization

The application of INT quantization allows CNN model size and computation to be reduced exponentially, making extreme quantization a hot research topic. However, firstly the decrease in bitwidth increases the probability of error in critical bits, which is unacceptable for safety-critical scenarios that require high resilience of CNNs. Secondly, extreme quantization often requires retraining with high overhead. Considering the higher tolerance of errors in high-bitwidth INTs and the local resilience differences of CNNs, we use mixed precision quantization (MPQ) to trade off resilience and compression effects. Existing MPQ methods mostly focus on compression as the first goal and lack indicators of resilience or vulnerability. Based on a large number of vulnerability analyses, we propose to use the maximum bit average offset and the approximation parameter average quantization error to measure layer vulnerability. And we measure the vulnerability of the entire CNN in terms of the accumulation of layer offsets. Accordingly, we design a complete vulnerability-aware MPQ (VulnA-MPQ) framework. From the perspective of minimizing vulnerability and compression ratio, VulnA-MPQ is performed on multiple CNNs using genetic algorithm (GA) with low overhead. The CNNs only need to be fine-tuned to restore accuracy after VulnA-MPQ. Experiments demonstrate that VulnA-MPQ not only maintains accuracy and compression ratio better but also improves CNNs fault tolerance. Within the 2% accuracy threshold, the random error data tolerated by VGG16 weights and activations are improved by 33%. The number of random errors tolerated by MobileNetV3-large weights is improved by 8.87 times, and that of activations is improved by 4.64 times.