A robust and efficient vehicle detection method from high resolution aerial image is still challenging. In this paper, a novel and robust method for automatic vehicle detection using aerial images over highway was presented. In the method, a GIS road vector map was used to constrain the vehicle detection system to the highway networks. After the morphological structure element was identified, we utilized the grayscale opening transformation and grayscale top-hat transformation to identify hypothesis vehicles in the light or white background, and used the grayscale closing transformation and grayscale bot-hat transformation to identify the hypothesis vehicles in the black or dark background. Then, targets with large size or covering a large area were sieved from the hypothesis vehicles using an area threshold that is much larger than a typical vehicle. Targets, whose width is narrower than the diameter of structure element utilized in the grayscale morphological transformation, were smoothed out from the hypothesis vehicles using binary morphological opening transformation. Finally, the hypothesis vehicles detected in both cases were overlaid. It should be noted that in the detection system, a vehicle could be detected twice by the two approaches. The two identical hypothesis vehicles should be amalgamated into a single one for accuracy assessment subsequently. We tested our system on seventeen highway scenes of aerial images with a spatial resolution of 0.15 × 0.15 m. The experimental results showed that the correctness, completeness, and quality rates of the proposed vehicle detection method were about 98%, 93%, and 92%, respectively. Thus, our proposed approach is robust and efficient to detect vehicles of highway using high resolution aerial images.