Nearest-neighbor-intersection algorithm for identifying strong predictors using high-dimensional data

Increasingly large amounts of data are being collected, but data quantity may fail to translate to predictability, when all data are affected by the same broad trends. That makes it important to identify the most important predictors for relevant outcomes. In this paper we assume that a large set of dependent attributes is available as well as a large set of independent ones, and that the data quantity is used for evaluating the predictability of the former based on the latter. This problem can be identified with the task of correlation analysis, which has been available for a long time. However, traditional correlation analysis techniques are not suited to the high-dimensional problems. We propose a nearest-neighbor-based approach for examining the strength of relationship between input and output. We demonstrate the effectiveness and efficiency of this approach for time series data that are cut in half so as to produce the two sets of related attributes.