This paper introduces a novel adaptive image segmentation algorithm which represents images by polygonal segments. The algorithm is based on an intuitive generative model for pixel intensities and its associated cost function which can be effectively optimized by a hierarchical triangulation algorithm. A triangular mesh is iteratively refined and reorganized to extract a compact description of the essential image structure. After analyzing fundamental convexity properties of our cost function, we adapt an information-theoretic bound to assess the statistical significance of a given triangulation step. The bound effectively defines a stopping criterion to limit the number of triangles in the mesh, thereby avoiding undesirable overfitting phenomena. It also facilitates the development of a multiscale variant of the triangulation algorithm, which substantially improves its computational demands. The algorithm has various applications in contextual classification, remote sensing, and visual object recognition. It is particularly suitable for the segmentation of noisy imagery.