Effective Data Structure for Smart Big Data Systems Applying an Expectation-Maximization Algorithm

In a general big data system there are two major factors that contribute to the processing delay. The total execution time consists of communication time and computation time. In most cases, modern big data systems offload to cloud computing systems to complete their job when too high of a computation load is being processed on the current task assigned systems. However, the adoption of cloud computing can cause the communication delay to increase. As big data analysis is used in more and more real-time systems, reducing the communication delay is one of the major challenges in big data research. This paper presents the application of an expectation-maximization algorithm for a big data system structure in order to reduce the communication time significantly. The fact that a decrease in the communication time in related to the gain of the information entropy is also revealed in this paper.