Big Data System Testing Method Based on Chaos Engineering

The big data systems usually have multiple interdependent network-elements and processes. With the gradual growth in the number of network-elements and the complexity of networking, the possibility of system abnormalities also increases, then the influences of faults on the system are difficult to assess. For effectively discovering the possible defects of big data system products and solving the stability problem of the evaluation system, chaos engineering is employed for the big data systems, and a big data system testing method based on chaos engineering is proposed. The proposed method, which is fault-tolerant, resilient and reliable, mainly includes the processes, i.e., exception injection, exception recovery, system correctness verification, observable real-time data statistics, and result reporting. The experimental results illustrate that the proposed big data system based on chaos method can shorten the development cycle of the version by 20% and observe 21 hidden faults caused by improper handling of abnormal scenarios, which effectively improves the robustness and stability of the distributed system.