EHANet: Efficient Hybrid Attention Network towards Real-time Semantic Segmentation

Semantic segmentation suffers from the contradiction between inference speed and model accuracy. State-of-the-art real-time methods improve inference rapidity by sacrificing feature representation and model capacity. This paper proposes a novel Efficient Hybrid Attention Network (EHANet) to remedy this dilemma. The EHANet follows an encoder-decoder structure, where the encoder is composed of Reduced Basic-Block (RBB) with very few parameters. At decoding stages, a hybrid attention mechanism is designed to re-weight the feature map. The attention mechanism employs contextual attention for deep features and spatial attention for shallow features. The proposed architecture makes a trade off between inference speed and segmentation performance. As a result, the proposed model achieves 66.1% mIoU on the Cityscapes validation set. Meanwhile, it can operate at a speed of 113 FPS on one NVIDIA Titan XP GPU for an input image with size of 1024×512.