Big Data analysis using Computational Intelligence and Hadoop: A study

Computational Intelligence (CI) techniques are expected to provide powerful tools for addressing Big Data challenges. The main techniques in CI, such as evolutionary computation, neural computation and fuzzy systems are inherently capable of handling various amount of uncertainty, which makes CI techniques well suited for dealing with Variability and Variety of Big Data. On the other hand, the other two V's, Volume and Velocity may create serious challenges to existing CI techniques. The next two V's that is Value ad Veracity are equally important and yet challenging in dealing with big data. Consequently, new CI techniques need to be developed to efficiently and effectively tackle huge amount of data, and to rapidly respond to changing situations. It should be pointed out, however, that such new techniques will not be developed from scratch; instead, they are based on many on-going research topics scattered in different areas of CI research, e.g., large-scale optimization, many-objective optimization, learning in non-stationary environments, and natural language processing. A recent review of the use of evolutionary computation and other meta-heuristics in optimization of biological systems indicate a similarity in imparting computational intelligence in huge amount of data while using biologically inspired techniques and with the big data analysis using Hadoop environment.