Latest Progress and Infrastructure Innovations of Big Data Technology

With the constantly emerging of new ways to release information on the internet, together with the rise of several other technologies such as cloud computing and the internet of things, data is explosively growing at an unprecedented speed, manifesting the coming of the era of big data. Facing the challenge of large-scale, demand diversity and real-time processing for big data, the efficient processing technology has become the core issue of research in the big data field. This paper elaborates in detail the key technologies dealing with big data processing as distributed computing technology, incremental processing technology, stream computing technology, etc., analyses and summaries the mainstream technology platform and system of big data represented by Hadoop, and finally introduces the latest infrastructure innovation progress driven by big data in the last part, including GPU acceleration technology, Software Defined Flash (SDF), etc..