In this paper, considering the adaptive characteristics of human visual system, a new color image enhancement algorithm based on human visual system adaptive filter is proposed. The new algorithm is divided into three major parts: obtain luminance image and background image, adaptive adjustment and color restoration. Unlike traditional color image enhancement algorithms, the adaptive filter in the algorithm takes color information into consideration. The algorithm finds the importance of color information in color image enhancement and utilizes color space conversion to obtain a much better visibility. Experimental results show that the algorithm proposed has better effectiveness in reducing halo and color distortion.