Ternary Weighted Networks with Equal Quantization Levels

Recent progress in deep convolutional neural networks has considerably changed the landscape of speech recognition, computer vision, natural language processing and so on. However, limited by the large number of parameters, the memory space and high computational complexity, it is a challenging task to deploy the deep neural network model in embedded system. To solve this problem, we propose Equal Trained Ternary Quantization (ETTQ), a ternary quantization method by improving Trained Ternary Quantization, which uses only a full-precision scaling coefficient for each layer, and quantize the weights to three levels. These positive and negative weights have same absolute values that are trainable parameters. Experiments show that the performance of ETTQ is only slightly worse than TTQ, but can converge faster and more stable during the training over CIFAR-10 and CIFAR-100 datasets.