Vanishing Point Detection and Rail Segmentation Based on Deep Multi-Task Learning

In modern railway systems, video surveillance and machine vision analysis have been widely used to detect perimeter intrusions. For pan-tilt-zoom (PTZ) cameras, the machine vision system needs to detect adjustments in PTZ cameras and then automatically determine the new alarm region in real time. In this paper, we propose a deep multi-task learning based algorithm for simultaneous vanishing point (VP) detection and rail segmentation, which can identify camera adjustment from changes in VP, and then automatically determine the alarm region from segmented rails. The multi-task based neural network consists of a feature extraction base network and three sub-task networks. The first sub-task network is a convolution regression network for VP detection. The second sub-task network utilizes an encoder-decoder structure for vanishing region (VR, a fixed region centered on VP) segmentation. The third sub-task network shares the encoder-decoder structure with the VR segmentation task and is used for rail segmentation. The VR segmentation task is activated only at the training stage, serving as an auxiliary task to enhance feature learning ability and increase VP detection accuracy. To further improve the accuracies of VP detection and rail segmentation, low-level features is modulated by high-level semantic information before feeding to the decoder stage. With the help of shared feature extraction and auxiliary training, the proposed VP prediction method needs very small training dataset and outperforms other methods in both efficiency and accuracy.