This paper studies a hierarchical approach for incrementally driving a nonholonomic mobile robot to its destination in unknown environments. The algorithm is modified to handle a map containing unknown information. Based on it, optimal (discrete) paths are incrementally generated with a periodically updated map. Next, accelerations in varying velocities are taken into account in predicting the robot pose and the robot trajectory resulting from a motion command. Obstacle constraints are transformed to suitable velocity limits so that the robot can move as fast as possible while avoiding collisions when needed. Then, to trace the discrete path, the system searches for a waypoint-directed optimized motion in a reduced 1-D translation or rotation velocity space. Various situations of navigation are dealt with by using different strategies rather than a single objective function. Extensive simulations and experiments verified the efficacy of the proposed approach.