Rethink the Linearizability Constraints of Raft for Distributed Key-Value Stores

Distributed key-value stores have been widely used as NoSQL systems or the storage layer of distributed relational databases for various big data applications (e.g., social networking, graph processing, machine learning, etc.) due to their excellent scalability and adaptability. Although modern hardware such as Flash-based SSDs and the high-speed network is commonly deployed in key-value stores to promote performance, the distributed consensus and consistency module (e.g., Raft) is typically the most time-consuming part in distributed systems. The reason lies in that Raft introduces some very strict constraints to ensure the linearizability. Therefore, in this paper, we rethink these constraints in-depth and find that some of them are not necessary, and can be broken to accelerate the performance significantly without breaking the linear consistency for distributed key-value storage systems. An improved distributed consensus algorithm called KV-Raft is proposed in this paper and implemented in an industry-level distributed key-value system, i.e., TiKV. The experimental results suggest that both the write and the read performance can be accelerated significantly by KV-Raft. For example, in the typical read/write-balanced case, KV-Raft promotes the system throughput by 53.6%, and reduce the average write and read latency by 37.8% and 29.4%, respectively.