A new approach for segmentation and classification of polarimetric synthetic aperture radar (POLSAR) data is proposed based on spectral graph partitioning. Since automated analysis techniques are often challenged due to the noisy properties of POLSAR data, human experts are employed to aid in the interpretation of such data in an operational setting. Humans can improve the performance of segmentation and classification of POLSAR data, because their vision system can apply cognitive skills that are not easy to incorporate into an automated system. The motivation for this paper is to incorporate some of these human perceptual skills into the computer algorithms. A framework that has recently emerged in computer vision for solving grouping problems with perceptually plausible results-spectral graph partitioning-is customized for POLSAR data. Segmentation is performed using the contour information in a region-based setting with the aid of spatial proximity. This is followed by a classification step performed through graph partitioning based on similarities of the mean coherence matrices obtained for each segment. Using the proposed approach, the results achieved are superior to the Wishart classifier. Automated parameter selection procedures are under development. This framework also suggests a way to accommodate different representations of polarimetric data and combine them with other information sources (e.g., optical imagery and digital elevation models).