Removal of shadows from a single image is a challenging problem. Producing a high-quality shadow-free image which is indistinguishable from a reproduction of a true shadow-free scene is even more difficult. Shadows in images are typically affected by several phenomena in the scene, including physical phenomena such as lighting conditions, type and behavior of shadowed surfaces, occluding objects, etc. Additionally, shadow regions may undergo postacquisition image processing transformations, e.g., contrast enhancement, which may introduce noticeable artifacts in the shadow-free images. We argue that the assumptions introduced in most studies arise from the complexity of the problem of shadow removal from a single image and limit the class of shadow images which can be handled by these methods. The purpose of this paper is twofold: First, it provides a comprehensive survey of the problems and challenges which may occur when removing shadows from a single image. In the second part of the paper, we present our framework for shadow removal, in which we attempt to overcome some of the fundamental problems described in the first part of the paper. Experimental results demonstrating the capabilities of our algorithm are presented.