An NVMe-oF Distributed Storage Design Based on Etcd

The emerging NVMe-oF (NVMe over Fabrics) protocol provides access to an SSD on another node over the network using NVMe commands, providing performance close to that of an SSD using the NVMe protocol locally. To further improve performance, the SPDK (Storage Performance Development Kit) proposes an NVMe-oF implementation in user mode. Compared to the native NVMe-oF I/O stack provided by the Linux kernel, the SPDK's user mode I/O stack can significantly reduce I/O latency, however, the NVMe-oF implementation in the SPDK only serves as a demonstration and does not implement cluster management and failover features. In this paper, we propose a design of multi-node NVMe-oF distributed storage based on Etcd technology, which enables bulk deployment and management of clusters with the help of Etcd's node discovery and management functions. It also utilizes Etcd's K-V database to persist configuration information for failover and recovery.