In this paper, spectral subtraction is successfully applied to image processing and to detect defects in the integrated circuit (IC) image. By utilizing the characteristics of many of the same chips in a wafer, three images with defects located in the same position and different chips are obtained. The defect images contain the spectrum of standard image without any defects. Spectral subtraction presented in the paper can extract the standard image from the three defect images. The algorithm complexity of spectral subtraction detecting defects is close to that of Fourier transform. After obtaining the standard image, the speed and accuracy of defects detection can be greatly enhanced using the detection method presented in the paper. Using the image gray-scale matching technology, impact of illumination on IC defect detection is solved. Experiments demonstrate that spectral subtraction is fast and accurate to defect detection in an IC image, and the method has high robustness for illumination.