Hyperspectral Pixel Unmixing With Latent Dirichlet Variational Autoencoder

We present a method for hyperspectral pixel unmixing. The proposed method assumes that 1) abundances can be encoded as Dirichlet distributions and 2) spectra of endmembers can be represented as multivariate normal distributions. The method solves the problem of abundance estimation and endmember extraction within a variational autoencoder setting where a Dirichlet bottleneck layer models the abundances, and the decoder performs endmember extraction. The proposed method can also leverage the transfer learning paradigm, where the model is only trained on synthetic data containing pixels that are linear combinations of one or more endmembers of interest. In this case, we retrieve endmembers (spectra) from the United States Geological Survey Spectral Library. The model thus trained can be subsequently used to perform pixel unmixing on “real data” that contains a subset of the endmembers used to generate the synthetic data. The model achieves state-of-the-art results on several benchmarks: Cuprite, Urban Hydice, and Samson. We also present a new synthetic dataset, OnTech-HSI-Syn-21, that can be used to study hyperspectral pixel unmixing methods. We showcase the transfer learning capabilities of the proposed model on Cuprite and OnTech-HSI-Syn-21 datasets. In summary, the proposed method can be applied for pixel unmixing in a variety of domains, including agriculture, forestry, mineralogy, analysis of materials, and healthcare. In addition, the proposed method eschews the need for labeled data for training by leveraging the transfer learning paradigm, where the model is trained on synthetic data generated using the endmembers present in the “real” data.