How to Improve the Deep Residual Network to Segment Multi-Modal Brain Tumor Images

Brain tumor segmentation plays an important role in diagnosing brain tumor. Nowadays, intense interest has been received in applying convolution neural networks in medical image analysis, but its performance is restricted by the limitation of the depth of the network. And how to accelerate the information propagation and make full use of all the hierarchical features in the network is also of vital importance. To address these problems, this paper proposed Deep Residual Dilate Network with Middle Supervision (RDM-Net), which combines the residual network with dilated convolution. It can solve the problem of vanishing gradient and increase the receptive field without reducing the resolution. During images processing, some information of regions of small tumor could be discarded, for its resolution is attenuated to a single pixel by continuously convolutional operations. Therefore, the spatial fusion block, consisting of a pixel discriminator and a region discriminator, has been designed to reserve the detailed information in the region of small tumor. It evaluates the relationship between this single pixel and its adjacent region to obtain the spatial structure information of brain tumors. Furthermore, the middle supervision block consisting of proposal pyramid and multi-hierarchical loss is proposed, which shortens the distance of information path and reduces cumulative errors during the network training. The proposal pyramid is inspired by the idea of boost learning, which fuses each proposal at multiple resolution level to ensure that the network produces better predictions. Multi-hierarchical loss combines the loss of intermediate proposal in the middle layers and the loss of prediction of the output layer to achieve the effect of middle supervision. The results of experiments illustrate that our framework can effectively propagate features of each layer and increase the diversity of information to enhance feature hierarchy for medical image recognition. Compared to other state-of-the-art methods, our framework has performed well in the BRATS2015 challenge. In summary, the main contribution in our paper is that this work is an early attempt to adopt the concept of “middle supervision” on multi-modal brain tumor segmentation.