i-NVMe: Isolated NVMe over TCP for a Containerized Environment

Non-Volatile Memory Express (NVMe) over TCP is an efficient technology for accessing remote Solid State Drives (SSDs); however, it may cause a serious interference issue when used in a containerized environment. In this study, we propose a CPU isolation scheme for NVMe over TCP in such an environment. The proposed scheme measures the CPU usage of the NVMe over TCP worker, charges it to containers in proportion to their NVMe traffic, and schedules containers to ensure isolated sharing of the CPU. However, because the worker runs with a higher priority than normal containers, it may not be possible to achieve CPU isolation with container scheduling alone. To solve this problem, we also control the CPU usage of the worker by throttling NVMe over TCP traffic. The proposed scheme is implemented on a real testbed for evaluation. We perform extensive experiments with various workloads and demonstrate that the scheme can provide CPU isolation even in the presence of excessive NVMe traffic.

I. Introduction

Recently, disaggregated storage has been widely deployed in modern data center to satisfy the increasing demand for huge data storage volume of ML and big data processing [1], [2]. Unlike traditional storage systems, such as direct-attached storage and storage area networks (SAN), in which capacity expansion is limited by physical drive bays and expansion chassis, disaggregated storage is a flexible and scalable solution since a number of storage devices can be allocated to any server using a high-speed network fabric.