Local Geometry Preserving Deep Networks For Featurizing High Dimensional Datasets

The quality of the features engineered to represent data during dimension reduction greatly impacts the performance of machine learning. Autoencoders are a popular deep-learning approach for generating lower-dimensional representations of complex, high-dimensional data. Unfortunately, features derived from Autoencoders do not preserve the local geometry of data. Thus, relationships between data may be lost during the featurization process, leading to lower-quality representations. To address this, we propose Deep Maximum Variance Unfolding (DMVU), a deep learning architecture for manifold-preserving dimension reduction. DMVU is a deep-learning extension of Maximum Variance Unfolding (MVU), a classic machine learning approach for dimension reduction. DMVU retains the local geometry preserving characteristics of MVU, while leveraging deep learning’s representational capacity and ability to manage large, high-dimensional datasets. This is achieved by DMVU’s novel loss function, which encourages the distance between neighboring points to be preserved while encouraging overall variance between data points. Our experimental study demonstrates that DMVU consistently outperforms state-of-the-art methods using the standard metrics of continuity and trustworthiness on 4 benchmark datasets.