This paper presents an environment-detection-and-mapping algorithm for autonomous driving that is provided in real time and for both rural and off-road environments. Environment-detection-and-mapping algorithms have been designed to consist of two parts: (1) lane, pedestrian-crossing, and speed-bump detection algorithms using cameras and (2) obstacle detection algorithm using LIDARs. The lane detection algorithm returns lane positions using one camera and the vision module “VisLab Embedded Lane Detector (VELD),” and the pedestrian-crossing and speed-bump detection algorithms return the position of pedestrian crossings and speed bumps. The obstacle detection algorithm organizes data from LIDARs and generates a local obstacle position map. The designed algorithms have been implemented on a passenger car using six LIDARs, three cameras, and real-time devices, including personal computers (PCs). Vehicle tests have been conducted, and test results have shown that the vehicle can reach the desired goal with the proposed algorithm.