Alignment of Overlapping Locally Scaled Patches for Multidimensional Scaling and Dimensionality Reduction

Data observations that lie on a manifold can be approximated by a collection of overlapping local patches, the alignment of which in a low-dimensional euclidean space provides an embedding of the data. This paper describes an embedding method using classical multidimensional scaling as a local model based on the fact that a manifold locally resembles a euclidean space. A set of overlapping neighborhoods are chosen by a greedy approximation algorithm of minimum set cover. Local patches derived from the set of overlapping neighborhoods by classical multidimensional scaling are aligned in order to minimize a residual measure, which has a quadratic form of the resulting global coordinates and can be minimized analytically by solving an eigenvalue problem. This method requires only distances within each neighborhood and provides locally isometric embedding results. The size of the eigenvalue problem scales with the number of overlapping neighborhoods rather than the number of data points. Experiments on both synthetic and real-world data sets demonstrate the effectiveness of this method. Extensions and variations of the method are discussed.