By Topic

Quality assessment of classification and cluster maps without ground truth knowledge

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
A. Baraldi ; Inst. di Studi su Sistemi Intelligenti per l'Automazione, ISSIA-CNR, Bari, Italy ; L. Bruzzone ; P. Blonda

This work focuses on two challenging types of problems related to quality assessment and comparison of thematic maps generated from remote sensing (RS) images when little or no ground truth knowledge is available. These problems occur when: (1) competing thematic maps, generated from the same input RS image, assumed to be available, must be compared, but no ground truth knowledge is found to assess the accuracy of the mapping problem at hand, and (2) the generalization capability of competing classifiers must be estimated and compared when the small/unrepresentative ground truth problem affects the RS inductive learning application at hand. Specifically focused on badly posed image classification tasks, this paper presents an original data-driven (i.e., unsupervised) thematic map quality assessment (DAMA) strategy complementary (not alternative) in nature to traditional supervised map accuracy assessment techniques, driven by the expensive and error-prone digitization of ground truth knowledge. To compensate for the lack of supervised regions of interest, DAMA generates so-called multiple reference cluster maps from several blocks of the input RS image that are clustered separately. Due to the unsupervised (i.e., subjective) nature (ill-posedness) of data clustering, DAMA provides no (absolute) map accuracy measure. Rather, DAMA's map quality indexes are to be considered unsupervised (i.e., subjective) relative estimates of labeling and segmentation consistency between every competing map at hand and the set of multiple reference cluster maps. In two badly posed RS image mapping experiments, DAMA's map quality measures are proven to be: (1) useful in the relative comparison of competing mapping systems; (2) consistent with theoretical expectations; and (3) in line with mapping quality criteria adopted by expert photointerpreters. Documented limitations of DAMA are that it is intrinsically heuristic due to the subjective nature of the clustering problem, and like any evaluation measure, it cannot be injective.

Published in:

IEEE Transactions on Geoscience and Remote Sensing  (Volume:43 ,  Issue: 4 )