Homogeneity Test for Confusion Matrices: A Method and an Example

The confusion matrix is the standard way to report on the thematic accuracy of geographic information data. Two widely adopted indices for thematic accuracy controls upon error matrix are the overall accuracy and the Kappa coefficient. Provided that a multinomial sampling, this work proposes a new method for testing the homogeneity of two independent thematic classifications which is based on whole error matrices. Specifically, a test function is proposed that uses as a test statistic the discrete Hellinger distance, which null distribution is approximated by bootstrapping. The performance of this approximation is evaluated by simulation, which confirms that it behaves properly, especially for small and moderate sample sizes. Finally an example of application is included.