Distributed Fuzzy Rough Set for Big Data Analysis in Cloud Computing

Fuzzy rough set based feature selection is a widely adopted technique for big data analysis. However, the high accuracy of this technique depends on all the data correlations, so that it always works in the centralized computing mode. With the increasing data volume, the centralized server, especially its computation capability and memory space, cannot afford the computing of fuzzy rough set. To enable the fuzzy rough set for big data analysis, in this paper, we propose the novel Distributed Fuzzy Rough Set (DFRS) based feature selection in cloud computing, which separates and assigns the tasks to multiple nodes for parallel computing. The key challenge is to maintain the global information on each distributed node without conserving the entire fuzzy relation matrix. We tackle this challenge by a dynamic data decomposition algorithm and a data summarization process on each distributed node. Extensive experiments based on multiple real datasets demonstrate that DFRS significantly improves the runtime and its feature selection accuracy is nearly the same as the traditional centralized computing.