The extensive use of digital cameras in creating an electronic database of document images, leads to a major problem in managing the vast amount of information. Many images include user-added marks, such as highlights and handwritten notes, which can be used in content based image retrieval. This paper presents a novel method for determining the existence of user-added marks in a given image and classification of existing marks. Two different approaches are integrated for the detection of marks: one based on color-spaces and the other on anomaly detection in text-dominant images. The proposed method achieved very successful results.