Massage-Passing Interface Cluster Bulid upon System Kernel Environment

With the age of Big Data coming, the three defining characteristics of Big Data-Volume, variety and Velocity, make Cloud Computing facing new challenges. In response to the demand of Big Data analytics, using distributed computing cluster to process vast amounts of data is a megatrend. In this paper, we discuss the performance of distributed computing clusters provided by the current cloud computing platforms. Found that for the Message-Passing Interface (MPI) cluster, which is used in Scientific Computing, such as astronomical, atmosphere, physical spectrum... etc., cloud computing platforms provide less relevant integration services. Which made MPI cluster make inefficient use of cloud computing resources and unable to exert its high computing performance. To improve these, we propose MPI cluster architecture makes efficient use of cloud computing resources. To break the limitation of constructing traditional MPI cluster computing environment, the MPI cluster uses Socket Interface-TCP/IP Server & Client to build Communication System as the message-passing channel, and simplifies Operating System computing resources management mechanism by Group Manager. As the separated way described above, the MPI cluster can resiliently grasp Operating System computing resources, and work well on the cloud computing platform, that computing resources are virtualization and flexible. In order to make The MPI cluster flexible dispatch computing resources on the cloud computing platform more easily, we adopt Kernel Distributed Computing Management (KDCM), proposed by Chiu and Huang. By its ability to unify manage and allocate computing resources, provides the MPI cluster has a path to flexible dispatch computing resources. As the goal of the MPI cluster: Running in Linux kernel driver, and loading on KDCM, we name it MPI Kernel Cluster (MPIKC). As MPIKC and KDCM fit tightly, they can efficient dispatch computing resources, exert its high computing performance, and provide Operating System-based computing environment on cloud computing platform. At the end, we verify the correctness of running MPIKC on KDCM, and use MPIKC to do distributed computing of high computing load, each computing unit used, enhances a 0.5~1 times performance. We proved that MPIKC fits well with cloud computing platform, and exerts its high performance as the advantage of distributed computing cluster.