Reliability analysis of Hadoop cluster System based on proportional hazards model

With the universal application of Hadoop cluster system, its reliability is drawing more and more attention from academia to industry. Recently, hardware load-sharing systems (LSSs) usually use accelerated life testing (AL T) models, which are not suitable for Hadoop cluster system whose reliability is significantly dependent on system software. Contrary to hardware which runs all the time after a system runs, software performs only when it is used. This paper presents an approach for Hadoop cluster system reliability and degradation process analysis, which is modeled as a nonhomogeneous Markov process (NHMH) composed of several non-homogeneous Poisson processes (NHPPs). Hadoop cluster system mainly includes namenodes and datanodes, so the paper mainly includes the analysis of the reliability and degradation process of the namenodes and data nodes. But the workload of the namenodes is different from the datanodes. The reliability and degradation process of the namenodes is related with the datanodes. The result solved from NHMH is a time-varying reliability and degradation process over Hadoop cluster system lifetime. Finally, the evaluation experiment shows the potential of the approach.