Intrinsic dimensionality estimation and dimensionality reduction through scale space filtering

Dimensionality reduction techniques are designed to exploit the fact that most high-dimensional datasets from the real world do not uniformly fill the hyperspaces in which they are represented but instead their distributions usually concentrate to nonlinear manifolds of lower intrinsic dimensions. However, when these techniques are applied directly to the initial degraded and noisy data, the assumptions on the possible statistical separation of real world classes do not, in the general case, hold. In this paper, we argue that scale space filtering, by denoising and simplifying effectively the initial dataset, ameliorate the way the properties of our observations are been encoded, strengthening, thus, the assumptions on the possible statistical separation of real world classes. Experimental results on real hyperspectral datasets demonstrate that appropriate vector-valued scale space filtering significantly contributes to the intrinsic dimension estimation and dimensionality reduction of high dimensional datasets.