Inspired by the keyword-based text filter, this paper proposes an image filter which detects the spam image by matching with user-specified image content. In this way, detecting image spam e-mail is converted into image matching process. Stable local feature detection and representation is a fundamental component of image matching algorithms. SIFT has been proven to be the most robust local invariant feature descriptor. In this process, SIFT algorithm is applied. The images are extracted with SIFT features, which are used to carry out the image matching work. Our experiments demonstrate that SIFT has a good performance in spam image recognition.