Linear features are difficult to be separated from complicated background in color scanned topographic maps, especially when the color of linear features approximate to that of background in some particular images. This paper presents a method, which is based on energy density and the shear transform, for the separation of lines from background. First, the shear transform, which could add the directional characteristics of the lines, is introduced to overcome the disadvantage that linear information loss would happen if the separation method is used in an image, which is in only one direction. Then templates in the horizontal and vertical directions are built to separate lines from background on account of the fact that the energy concentration of the lines usually reaches a higher level than that of the background in the negtive image. Furthermore, the remaining grid background can be wiped off by grid templates matching. The isolated patches, which include only one pixel or less than ten pixels, are removed according to the connected region area measurement. Finally, using the union operation, the linear features obtained in different sheared images could supplement each other, thus the lines of the final result are more complete. The basic property of this method is introducing the energy density instead of color information commonly used in traditional methods. The experiment results indicate that the proposed method could distinguish the linear features from the background more effectively, and obtain good results for its ability in changing the directions of the lines with the shear transform.