This paper presents a new approach to represent an image by "verge points," which are defined as high-curvature points on the image surface. This representation offers a compact and reversible way to preserve the essence of the original image. Various applications, such as compression, edge detection, image enhancement, and image editing, can be achieved based on this representation. In this paper, the whole procedure for verge point representation is presented. Based on these verge points, image reconstruction can be easily achieved via iterative linear interpolation. These extracted verge points with compatible properties are further linked into verge curves to offer more compact representation. Progressive representation is also developed based on a multiscale extraction scheme. Some potential applications are then presented to demonstrate the versatility of this representation.