Minimum value sampling algorithm based on CDC

The explosive growth of data increases the burden of storage systems. At the same time, the redundant data result in a waste of storage space and an increase in cost. Data deduplication is urgently required to alleviate these problems. Similarity detection is very critical to the performance of data deduplication. This technique reduces memory consumption and improves the data deduplication ratio by sampling. In this paper, we improve the Enhanced Position-Aware Sampling algorithm (EPAS), proposing minimum value sampling algorithm based on Content-Defined Chunking (CDC), which introduces the sampling algorithm into the data chunking process, continuously calculating the hash value of the sliding window and then selecting the minimum value as a sample. The experiment shows that data deduplication ratio of the proposed algorithm is higher than EPAS algorithm at the same sampling ratio.