A Vision-Based Approach for Rail Extraction and its Application in a Camera Pan–Tilt Control System

Rail extraction, i.e., determining the position of the rails ahead of a train, is one of the basic tasks of vision-based driver support in railways. This paper introduces an approach that extracts rails by matching edge features to candidate rail patterns modeled as sequences of parabola segments. Patterns are precomputed in a semiautomatic offline stage for areas near the camera and generated on the fly for more distant regions. Our approach was designed to address the challenges posed by the open environment without requiring explicit knowledge about train speed or camera parameters/position and running fast enough for practical use without specialized hardware. Evaluation was performed on hours of video captured under real operation conditions, considering the requirements of a system in which a camera with zoom lens mounted on a pan-tilt unit captures images from the area ahead with increased resolution.