This study presents a hierarchical lane detection system with the ability to deal with both structured and unstructured roads. The proposed system classifies the environment first before applying suitable algorithms for different types of roads. Instead of dealing with all situations with one complicated algorithm, this hierarchical architecture makes it possible to achieve high accuracy with relatively simple and efficient lane detection algorithms. For environment classification, pixels with lane-marking colours are extracted as feature points. Eigenvalue decomposition regularised discriminant analysis is utilised in model selection and maximum likelihood estimation of Gaussian parameters in high-dimensional feature space. For structured roads, the extracted feature points are reused for lane detection. Moving vehicles that have the same colours as the lane markings are eliminated from the feature points before the authors perform angles of inclination and turning points searching to locate the lane boundaries. For unstructured roads, mean-shift segmentation is applied to divide the scene into regions. Possible candidate pairs for road boundaries are elected from the region boundaries, and Bayes rule is used to choose the most probable candidate pairs as the lane boundaries. The experimental results have shown that the classification mechanism can effectively choose the correct lane detection algorithm according to the current environment setting, and the system is able to robustly find the lane boundaries on different types of roads in various weather conditions.