Efficient Decoder Reduction for a Variety of Encoder-Decoder Problems

Encoder-decoder networks have become the standard solution for a variety of segmentation tasks. Many of these approaches use a symmetrical design where both the encoder as well as the decoder are approximately of the same computational complexity. However, symmetrical properties of encoder-decoder networks are not necessarily optimal. This work proposes an elegant and generic method to reduce the decoder complexity in encoder-decoder networks by scaling the number of feature channels in the decoder. The popular network U-Net is used as an example for how to adapt existing models with symmetrical properties. The effect of the decoder size is investigated on three data sets with varying complexity, namely, the ISIC, Cityscapes, and SUN RDB-D data sets. We show that a reduction in decoder channels shows no statistically differing results while at the same time providing a decoder requiring up to 99% fewer FLOPs, (± 90% fewer FLOPs is attainable for all investigated problems). In addition, results show that the number of parameters in the decoder of two models which already have smaller decoders can be further optimised depending on the problem. The proposed solution is a simple method and can easily be implemented in other encoder-decoder models. Empirical results also show that a reduction in parameters may even lead to improved performance, which is likely due to fewer parameters, reducing overfitting effects.