Distributed Storage Codes Based on Double- Layered Piggybacking Framework

Piggybacking codes are a class of distributed storage codes adopted in distributed storage systems, which received much attention in recent years. Two novel piggybacking constructions based on the double-layered piggybacking framework are proposed. They can reduce the repair bandwidth of both the systematic nodes and the parity nodes efficiently. The average repair bandwidth ratio for the two constructions approaches to zero asymptotically as the number of systematic nodes and parity nodes tends to infinity. Compared with other piggybacking codes, the two proposed double-layered piggybacking codes, especially the second one, not only require less design constraints, but also obtain the optimal comprehensive repair efficiency, which further save the amount of data read and downloaded.


I. INTRODUCTION
With the advent of the 5G era, applications have become more and more abundant, and the number of users has also increased, which has led to a geometric growth of data [1]. The performance and scalability of storage systems are posing a serious challenge due to the large scale and the rapid growth of data. In the traditional centralized storage systems, with large scale and rapid growth of data, the burden of the storage servers has been the bottleneck of the system, which makes the reliability and security untenable and they cannot meet the requirement of big data applications [2]. At the same time, the distributed storage systems receive much attention because of their higher reliability and stronger scalability. However, various kinds of malfunctions, such as power failure, system maintenance, network interruption, etc., will occur frequently due to the existence of a large number of separated nodes [3]. In order to retain the reliability of the whole storage system, replication and erasure codes [4] are used as the two common fault-tolerant techniques. Replication is the simplest redundant fashion and is used in Hadoop Distributed The associate editor coordinating the review of this manuscript and approving it for publication was Feng Li . File System, Google File System, Windows Azure, etc.. But it provides low storage efficiency. On the contrary, erasure codes introduce some redundancy obtaining much higher storage efficiency and reliability. They are adopted by many distributed storage systems such as Google Colossus, HDFS Raid, Microsoft Azure, OceanStore, and so on [5], [6].
To retain the fault tolerance of the whole storage system and the reliability of data downloaded, the failed nodes need to be repaired in time. A good fault-tolerant technique requires that the storage system has low redundancy overhead and low node repair bandwidth. Consequently, how to reduce the repair bandwidth of a failed node has become a research hotspot in the study of distributed storage codes. For this purpose, several distributed storage codes with low repair bandwidth are presented, such as the regenerating codes [7]- [11] and the local repairable codes (LRCs) [12]- [15]. But their construction procedures are very complex due to matrix operations during the repair process. Furthermore, MDS (maximum distance separable) array codes, such as EVENODD codes [16], [17], B-code [18], X-code [19], RDP [20], [21], STAR [22] and Zigzag codes [23], etc. with low complexity are presented. Their encoding and decoding procedures are basically based on VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ the simple operation of low-order finite field. Due to the construction procedures of MDS array codes, they have fewer redundant nodes, which makes the fault-tolerant capability limited and the repair efficiency low. In 2013, K.V. Rashmi et al. proposed the idea of piggybacking framework to reduce the repair bandwidth for node repair in distributed storage systems [24]. The proposed piggybacking framework is based on an MDS code where data symbols in the instances are embedded into the piggyback blocks in other instances via the well-designed piggyback functions. The missing data symbols can be recovered by solving the piggyback functions instead of decoding the traditional MDS code, which can reduce the repair bandwidth effectively. The piggybacking codes have a distinguished feature that they can reduce the extra storage overhead without changing the node distribution structure. No additional node is added except the parity check nodes of MDS codes. Moreover, satisfying the properties of MDS codes, the piggybacking codes can achieve high-rate, have small number of substripes, and support any codeword length n or information length k of the base code (the value of parameters is only depended on the requirement of the base code). Some codes such as the Rotated-RS codes must satisfy that the number of parity nodes r ∈ {2, 3} and the number of systematic nodes k ≤ 36 [25], and the EVENODD as well as RDP codes only can be achieved at r = 2. The piggybacking codes have been adopted by Facebook Warehouse Cluster, Hadoop Distributed File System (HDFS) and other systems [26] for their flexible design, low complexity and convenient implementation.
Since the invention of piggybacking codes, researchers are trying to find effective piggybacking construction method to repair the failed nodes. In [27], S. Yuan et al. introduced generalized piggybacking codes to repair the systematic nodes and proved that the average repair bandwidth ratio of systematic nodes can approach to zero with the number of parity nodes tending to infinity, which is very close to the cut-set bound of distributed storage codes. In [28], S. Kumar et al. also investigated a piggybacking framework for repairing systematic nodes, which is based on two classes of parity nodes. The primary goal of the first class of parity nodes is to provide good fault tolerance, while the second is to reduce the repair bandwidth and the repair complexity. With the number of the second class of parity nodes increasing, the repair bandwidth of the systematic nodes can be reduced, sacrificing the storage overhead of the system. C. Shangguan et al. presented a piggybacking construction for systematic nodes in [29], in which the systematic nodes are divided into irrelevant subsets with various piggyback blocks. Systematic nodes in different subsets have different repair bandwidth. Thus higher repair efficiency can be obtained and little work is done on the construction design for repairing parity nodes. B. Yang et al. considered a piggybacking framework to repair parity nodes and preserve the repairability of systematic nodes at the same time in [30].
It is worth mentioning that a large-scale distributed storage system will cost a large amount of network bandwidth if a failed node completely relies on the cloud center to be repaired. At the same time, the bottleneck of the whole system performance becomes the bottleneck of the network bandwidth, because the processing speed of the cloud center is much more than the speed of data transmission in the network. Obviously, if the computing tasks of the node repair can be completed at the edge nodes and only the result is submitted to the cloud center, not only can the unnecessary bandwidth be saved, but also the response time can be reduced, which improves the performance of system tremendously. In addition, performing edge computing tasks at the local nodes can greatly reduce the risk of data being leaked in network transmission, which can effectively protect data security and privacy.
As shown in Fig. 1, a collaborative edge frame, which connects the multiple edge nodes that are geographically distributed despite their physical location and the network structure is proposed in [31]. Edge node is a physical small data center that connects users and cloud with data processing capability. The requirement for data processing will be transmitted between the cloud and the edge nodes, and the data will be transmitted between the users and the edge nodes. The edge nodes can not only request service and content from the cloud but also perform the computations locally, such as computing offloading, data storage, caching and processing, along with IoT management etc. In view of those abilities, the edge nodes need to be well-designed to achieve the goal of data reliability, security, and privacy protection [32].
From what has been discussed above, it is safe to infer that these distributed storage edge nodes provide the opportunity to process and share data for users. Therefore, Edge computing has the significantly potential to address the concern of repairing failed nodes in distributed storage systems with piggybacking codes.
In this paper, we propose two kinds of novel double-layered piggybacking constructions that can reduce the repair bandwidth of both the systematic nodes and the parity nodes efficiently. Asymptotically, the average repair bandwidth ratio approaches to zero as the number of systematic nodes and parity nodes tends to infinity. Compared with other piggybacking codes, the proposed schemes, especially the second one, require less design constraints while having the optimal comprehensive repair performance, and further save the amount of data read and downloaded during the node repair process.
The remainder of this paper is organized as follows. In section II the base code and the piggybacking framework are introduced briefly. In Section III, a novel double-layered piggybacking framework is proposed. Based on the proposed scheme, an improved double-layered piggybacking framework is presented in Section IV. The decoding complexity is given in Section V. Subsequently, the comparison of the two proposed double-layered piggybacking codes with other piggybacking codes is given in section VI. In Section VII, the double-layered piggybacking codes with multiple instances are introduced. Finally, the conclusion is given in Section VIII.

A. THE BASE CODE
Generaly, the piggybacking framework can be operated on an arbitrary existing code, which is termed the base code. We use MDS (Maximum Distance Separable) codes as the base code in the proposed piggybacking framework because MDS codes can provide the optimal trade-off between the storage and the reliability [33]. In a distributed file storage scheme, the original file is initially divided into k data blocks. Subsequently, they are encoded into n blocks and preserved by n distributed storage nodes. We can recover the original file as long as accessing any l(l ≥ k) nodes via erasure decoding. If l = k, the base code satisfies the MDS property of tolerating the failures of any n − k storage nodes at most [34]- [36].
As shown in Fig. 2, the original file is firstly divided into k blocks {d 1 , d 2 , . . . , d k } with length m via serial-to-parallel conversion. The ith elements of all the blocks are arranged together to form the information bits of the substripes a i (1 ≤ i ≤ m). The k information bits in each substripe are encoded into n = k + r bits using the MDS encoding functions f 1 , f 2 , . . . , f k+r obeying the generator matrix of the (n, k) MDS code. The ith (0 ≤ i ≤ n) elements of all the m substripes are stored in the node i.

B. THE PIGGYBACKING FRAMEWORK
The piggybacking framework operates on m instances of the base code (we use the MDS code) and well-designed piggyback functions are introduced to add the information of some instances to other ones. The piggybacking framework adopts the systematic encoding form. Since the systematic nodes store all the original data symbols, the users can directly download the original data from the data symbols without using the decoding algorithm. Thus, the complexity and delay of data recovery are greatly reduced, which makes the information transmission more efficient.
Clearly, the first column of the framework is identical to a codeword of the base code. We can get a 1 directly using the decoding procedure of this base code. In the second instance, subtracting the piggyback functions {g 2,i (a 1 )} n i=1 from the elements of the instance, the message vector a 2 can also be decoded via the base code decoding. Continuing this way, for any instance i, (2 ≤ i ≤ m), the piggyback functions can be subtracted and the base code of that instance can be obtained, which can be decoded using the base code decoding. In particular, we may enable piggybacking codes to achieve our desired goals (eg. lower the repair bandwidth) by designing flexible piggyback functions.

III. DOUBLE-LAYERED PIGGYBACKING CODES
Distributed storage systems provide data to users quickly and effectively relying on the availability provided by systematic nodes, and the systems retain this availability depending on the reliability provided by parity nodes. Neither of the two kinds of nodes can exist independently in distributed storage systems for a long time. Whereas the existing piggybacking codes are designed mostly for either repairing the systematic nodes or repairing the parity nodes. We propose a double-layered piggybacking framework that can efficiently repair both the systematic nodes and the parity nodes at the same time. The first layer of the piggybacking design is to repair systematic nodes, and the second layer is for repairing parity nodes. We introduce the construction method using a simple example firstly, then we generalize it to the general condition.
An erasure code consists of n units, where k of these units are systematic units, and r are parity units. We call this set VOLUME 8, 2020 of n = (k + r) units a stripe. In a distributed storage system, a node stores a large number of systematic and parity messages belonging to different stripes. Each stripe is conceptually independent and identical. Without loss of generality, we will consider only one stripe and each stripe can be viewed as consisting of multiple substripes. The number of substripes in a stripe is equal to the size of the vector that constitutes a unit of the code.

1) CODE DESIGN
To repair the systematic nodes, the substripes 3 and 4 are selected as reserved substripes for embedding the data symbols into piggyback blocks. All the symbols except the two substripes are embedded into the latter 4 parity nodes on the two reserved substripes.
To ensure that each systematic node can be recovered successfully, the data symbols in a systematic node cannot be embedded in the same piggyback block. Therefore, the data symbols need to be dispersed as much as possible [27]. According to the analysis, when the number of piggyback blocks is a constant, the number of data symbols embedded in every piggyback block directly affects the repair bandwidth of systematic nodes. So, we choose the best solution and adopt the general idea [24], [27], [29], [37]. In our example, the embedding method is explained as follows.
The data symbols to be embedded are arranged sequentially into a row from left to right in order, i.e., f 4 . Then they are embedded into the piggyback blocks in each parity node one by one. In our example, the two reserved substripes of the first parity node do not participate in piggyback functions because they are the origination of repairing systematic nodes. The first embedded data symbol is embedded in the first piggyback block. Then the following symbol is embedded in the next piggyback block. When each piggyback block has one data symbol and there are data symbols left, we will return to the first piggyback block to embed the data symbol in order till all the data symbols are embedded.
To repair the parity nodes, the two backmost substripes are selected for embedding piggyback blocks. The parity symbols of all the parity nodes are embedded in the two substripes in a specific way. We regard each piggyback block in parity nodes and its corresponding parity symbol of substripes 3 and 4 as a ''black box'' B, which is embedded in piggyback blocks again in the substripes 5 and 6, e.g., B 3 ). Using the ''black box'', we don't care about which data symbols or parity symbols are in it but just treat the ''black boxes'' as ordinary parity symbols.
Similar to the data symbols, the parity symbols also need to be dispersed as much as possible. One difference is that the repair of parity nodes depends on the parity nodes associated with them. Therefore, it is necessary that the symbols of a parity node cannot be embedded into the piggyback blocks of the node itself. If the parity node fails, the symbols and the piggyback block associated with them are also lost, then the symbols cannot be solved by piggyback functions. Thus, we use a diagonal-order method as shown in Fig.4 to form the piggyback blocks in substripe 5 and 6, which can avoid the scenery we mentioned.
A few notes on the code design of the double-layered piggybacking framework are given as follows.
• Firstly, the reserved substripes of systematic nodes and the piggybacked substripes of parity nodes are not overlapped in this example. If the two types of substripes are overlapped, to repair the systematic nodes, what we need to read are not only the data symbols embedded in the piggyback blocks, but also the parity symbols embedded. It is clear that the non-overlapped design needs less repair bandwidth than the overlapped design for repairing the systematic nodes. To repair the parity nodes, the symbols need to be read are not only the parity symbols embedded in the piggyback blocks, but also the data symbols embedded. The number of data symbols is more than the number of ''black boxes''.
Obviously, the non-overlapped design needs less repair bandwidth than the overlapped design for repairing the parity nodes, too. It is because that the ''black boxes'' hide the symbols read unnecessarily and ensure that no extra repair bandwidth will be introduced when the two classes of nodes are repaired. Moreover, only in non-overlapped design, can ''black boxes'' exist. Hence, we use a non-overlapped piggybacking design.
• Secondly, we analyze the case of whether the parity symbols on the reserved substripes of the first parity node (i.e., f (3) k+1 and f (4) k+1 in this example) are embedded in piggyback blocks or not. If the two parity symbols are not embedded, they can only be recovered by the decoding process of base codes. On the contrary, suppose that they are embedded in piggyback blocks, they can be recovered via solving the piggyback functions, which increases the number of symbols in the piggyback blocks unfortunately, resulting in the increase of the repair bandwidth for the other symbols. However, when the number of these symbols is small, the two methods have little influence on the performance of the whole framework. In order to understand the two-layered piggybacking framework we proposed, the embedding method is adopted in this paper.
• Finally, there is another method to design the piggyback blocks for parity nodes. We place all the parity symbols and ''black boxes'' according to the order of the matrix arrangement: starting from the first row of the first column of the matrix to the last row, then to the first row of the second column until all the elements of the matrix are embedded. If the number of parity nodes is odd, the number of parity nodes is symmetrical about the middle number. In this way, parity symbols in a parity node are easily embedded in the piggyback blocks of this parity node, and we have to make additional adjustments. Consequently, we adopts the diagonal-embedding method. The diagonal method can also be used to construct piggyback blocks for systematic nodes.

2) REPAIR ANALYSES
Assume that the first systematic node fails. All the data symbols {f i } 4 i=2 and the parity symbols {f k+1 } on the reserved substripes are read to recover the missing data {f 1 } via the MDS code decoding. The amount of data read in this step is 8. {f k+3 } and the data symbols except the missing ones embedded in them, we can recover the missing data symbols {f 1 } via solving piggyback functions. The amount of data read in the second step is 8. Hence, the total amount of data read is 16. The repair strategy of other systematic nodes is similar.
Suppose that the parity node with index k + 2 fails. All the data symbols {f The amount of data read in this step is 8. Next, read the piggyback blocks in which the missing symbols (including the parity symbols {f k+2 }) are embedded and read the symbols embedded in these piggyback blocks except the missing ones. The missing symbols can be recovered by solving the piggyback functions. The amount of data read in this step is 8. Finally, the symbols {f k+2 , f (6) k+2 in step1) are downloaded to repair the piggyback block of the failed parity node. The amount of data read in this step is 4. The total amount of data read is 8 + 8 + 4 = 20. The repair strategy of other parity nodes is similar.

B. THE GENERAL DOUBLE-LAYERED PIGGYBACKING FRAMEWORK
Consider s substripes of a (k + r, k) systematic MDS code as the base code. The number of reserved substripes for repairing systematic nodes is a and the number of piggybacked substripes for repairing parity nodes is b.
are the piggyback blocks embedding data symbols and they are also the ''black boxes''. {P are the piggyback blocks embedding parity symbols and ''black boxes''. The embedding methods of the two classes of piggyback blocks are the same as the methods in the (9, 4) code given before. We obtain a general double-layered piggybacking framework, refer to Fig.5.

1) REPAIRING SYSTEMATIC NODES
In the general double-layered piggybacking framework, there are a(r − 1) piggyback blocks (or black boxes) in the a reserved substripes. Without loss of generality, specify the a(r − 1) piggyback blocks contain n 1 , n 2 , · · · , n a(r−1) data symbols respectively. Obviously, is the number of data symbols being embedded. Suppose that the ith systematic node fails, i ∈ {1, 2, · · · , k}, the repair strategy of reconstructing the missing symbols

Algorithm 1 Repairing the Systematic Nodes
Step 1: Read all the surviving data symbols j=1, =1 , j = i in the a reserved substripes, and the parity symbols The a missing data symbols {f in the reserved substripes can be recovered via the MDS code decoding.
Step 2: Read the piggyback blocks where the remaining missing data are embedded and the other data symbols in these piggyback blocks except the missing data symbols. Then we can get the remaining s−a missing data symbols by solving the piggyback functions.
The remaining s − a missing data symbols are embedded in s − a piggyback blocks respectively. Let the number of data embedded in these blocks be {n ij } s−a j=1 respectively. The number of extra data symbols (in the same piggyback blocks except the missing data symbols) needing to be read is s−a j=1 n ij − s + a. The amount of data read in Step 1 is ka, and there are k systematic nodes. So in this part, the repair bandwidth is k 2 a. Consider the ith piggyback block, 1 ≤ i ≤ a(r − 1), VOLUME 8, 2020 and suppose the piggyback block contains n i data symbols. In the repair process, this piggyback block needs to be downloaded n i times. The amount of data read is For each data symbol in this piggyback block, it need be downloaded n i − 1 times. Therefore, the amount of data read is . Then, the repair bandwidth of all the k systematic nodes is The average repair bandwidth ratio of systematic nodes η sys D1 ). (2) Define the following three variables, where u l is the quotient and u is the remainder of k(s−a) a(r−1) . Being the number of almost evenly embedded data symbols, n i − n j is either 1 or 0. Thus In what follows, we obtain a lower bound and an upper bound of η sys D1 . The discussions of this issue are as follows. ) is a multiple of a(r − 1), i.e., u = 0, the data symbols are embedded in each piggyback blocks evenly. By Eq. (4), we can have Define p a = a/s being the proportion of reserved substripes. It holds that Hence, a lower bound of η sys D1 for the double-layered piggybacking codes is Obviously, low (η sys D1 ) is a function of p a and the number of parity nodes r.
is not a multiple of a(r − 1)a, i.e., u = 0, data symbols embedded in each piggyback blocks are uneven. According to the inequality of arithmetic and geometric means, it is easy to get Consequently, by Eq. (4), we can have Because p a = a/s is the proportion of reserved substripes. It holds that Hence, a upper bound of η sys D1 for the double-layered piggybacking codes is Clearly, up (η sys D1 ) is a function of p a and the number of both systematic nodes k and parity nodes r. Suppose one case that the code rate of the base code is 0.5, the curves of low (η  r equals to 5, 10, 20, 100 and 1000 respectively are illustrated in Fig. 6.
The results in Fig. 6 suggest that the curves of low (η

2) ANALYSES OF REPAIRING PARITY NODES
For the general double-layered piggybacking framework, there are b · r piggyback blocks in the b piggybacked substripes. Without loss of generality, define the b · r piggyback blocks contain m 1 , m 2 , · · · , m b·r symbols (including the parity symbols and the ''black boxes''). b·r j=1 m j = r(s − b). r(s − b) is the number of the parity symbols and the ''black boxes'' that need to be embedded. Suppose that the rth parity node fails, we use the following algorithm to reconstruct the missing symbols in (12), as shown at the bottom of this page.
The method to recover the missing symbols which are in the non-piggybacked substripes is similar to recovering a systematic node. The difference is that the piggyback blocks in the piggybacked substripes of the failed parity node are also needed to be reconstructed. The amounts of data read in Step 1 and Step 3 are k · b and s − b respectively. The total repair bandwidth of all the r parity nodes is Algorithm 2 Repairing the Parity Nodes Step 1: Read all data symbols {f of the b piggybacked substripes and calculate the parity symbols {f on the piggybacked substripes.
Step 2: Download the piggyback blocks where the missing parity symbols and ''black boxes'' are embedded and the other symbols in the piggyback blocks except the missing parity symbols and ''black boxes''. We can get the s − b missing symbols by solving the piggyback functions.
Step 3: Reconstruct piggyback blocks (which contain the parity symbols and ''black boxes'') of the failed parity node, the remaining b missing symbols are recovered completely.
The average repair bandwidth ratio of parity nodes η par D1 is Define the following three variables, where v l and v are the quotient and remainder of r(s − b)/br respectively. By Eq. (14), we have We obtain a lower bound and an upper bound of η par D1 as follows.
• If r(s − b) is a multiple of br, i.e., v = 0. By Eq. (16), we have VOLUME 8, 2020 Define p b = b/s being the proportion of piggybacked substripes. It holds that Hence, a lower bound of η par D1 for the double-layered piggybacking codes is Obviously, low (η par D1 ) is a function of p b and the number of systematic nodes k.
is not a multiple of br, i.e., v = 0, according to the inequality of arithmetic and geometric means, Consequently, by Eq. (16), we can get As Hence, a upper bound of η par D1 for the double-layered piggybacking codes is which is a function of p b and k. The curves of low (η par D1 ) and up (η par D1 ) with different p b for k equals to 5, 10, 20, 100 and 1000 respectively are illustrated in Fig. 7.
The results of Fig. 7 suggest that the curves of low (η   (both the systematic nodes and the parity nodes) η all D1 based on this framework is obtained.
The partial derivatives of low (η sys D1 ) and up (η sys D1 ) with respect to p a are as follows, It can be obtained that p a = √ r r and p a = The curves of min( low (η sys D1 )) and min( up (η sys D1 )) with different r for code rate equals to 0.4, 0.5, 0.6 and 0.8 respectively are illustrated in Fig. 8. • min( low (η sys D1 )) is dominated by the number of parity nodes r only, having no relationship with the code rate.
• min( up (η sys D1 )) is related to the number of both parity nodes r and systematic nodes k. Moreover, the higher the code rate, the smaller the min( up (η sys D1 )). • The curves of min( low (η sys D1 )) and min( up (η sys D1 )) are closer to each other with increasing r. There exists a double-layered piggybacking code whose repair bandwidth ratio of systematic nodes is extremely close to min( low (η sys D1 )). The partial derivatives of low (η par D1 ) and up (η par D1 ) with respect to p b are as follows, It can be obtained that when (28) and (29) equal 0, respectively. Being convave functions about p b , the min( low (η par D1 )) and min( up (η par D1 )) are as follows, for p a = √ r r and p b = √ k k . The asymptotic analyses of min( low (η all D1 )) are obtained as shown in (34), (35), and (36), as shown at the bottom of the next page. Consequently, we can get the following conclusion.
• When k, the number of systematic nodes, tends to infinity, the min( low (η all D1 )) approaches to a constant, which is related to r.
• When r, the number of parity nodes, tends to infinity, the min( low (η all D1 )) approaches to a constant, which is related to k.
• When both k and r tend to infinity, the min( low (η all D1 )) approaches to zero. The variation of min( low (η all D1 )) vs k and r is shown in Fig.10. Obviously, when k and r continue increasing, VOLUME 8, 2020 min( low (η all D1 )) of the proposed double-layered piggybacking framework approaches to zero infinitely. The repair efficiency is higher with lower bandwidth ratio. In particular, our design is very suitable for large-scale distributed storage systems.

IV. THE IMPROVED DOUBLE-LAYERED PIGGYBACKING FRAMEWORK
In this section, the improved double-layered piggybacking framework is proposed. Based on the double-layered piggybacking framework presented in the former section, the construction of the piggyback functions for repairing systematic nodes is changed. Compared with the former framework, the improved method achieves a lower repair bandwidth ratio of systematic nodes when the number of parity nodes is over 3. Consequently, for the whole distributed storage system, the repair bandwidth ratio is lowered for all kinds of nodes, which is equivalent to a higher comprehensive repair efficiency. We introduce the improved method via a simple example firstly, and then we generalize it.

1) CODE DESIGN
Initially, all the systematic nodes in the piggybacking framework are divided into three subsets, i.e., 1 = {1, 2}, 2 = {3, 4}, and 3 = {5, 6}. Subsequently, these 3 subsets of data symbols are embedded into the reserved substripes 5, 4, and 3, respectively. For the 5th substripe, the data symbols in the first subset 1 except the ones in the 5th substripe are embedded. For the 4th substripe, the data symbols in the second subset 2 except the ones in the 5th and 4th substripes are embedded. For the 3rd substripe, the data symbols in the third subset 3 except the ones in the 5th, 4th, and 3rd substripes are embedded. The key idea of this construction is dividing the systematic nodes into several subsets and the piggyback blocks (black boxes) in a substripe embedding part of the data symbols in the same subset. Moreover, the data symbols in a substripe of a subset are embedded into the same black box. The diagonal-order method (refer to sections III) is adopted to repair parity nodes in this improved double-layered piggybacking framework too.

2) REPAIR ANALYSES
The amount of data read and downloaded is different when repairing failed nodes in different subsets because of the different number of data symbols embedded in piggyback blocks in different subsets. Taking one node of each i (1 ≤ i ≤ 3) as example, the repair process is as follows.
• Suppose that the first systematic node fails. The data symbols {f (5) j } 6 j=2 and parity symbol f (5) k+1 are read to recover the missing data symbol f  Hence, the total amount of data read and downloaded is 18. The repair strategy of systematic node with index 2 is similar.
• Suppose that the third systematic node fails. The data symbols {f ) are read to recover the missing data symbol (f 3 ) via the MDS code decoding. The amount of data read in this step is 12. {f lim r→∞ min( low (η all D1 )) = lim  • Suppose that the fifth systematic node fails. The data symbols {f ) are read to recover the missing data symbols (f

B. THE GENERAL IMPROVED DOUBLE-LAYERED PIGGYBACKING FRAMEWORK
Using a (k + r, k) systematic MDS code as the base code to form s substripes. The number of piggybacked substripes for repairing parity nodes is b. All systematic nodes are divided into L subsets { 1 , 2 , · · · , L }, the number of systematic nodes in subset The improved general double-layered piggybacking framework we proposed is shown in Fig.12 are the piggyback blocks embedding data symbols to repair systematic nodes. {P ,j=1 are the piggyback blocks embedding parity symbols and ''black boxes'' to repair parity nodes. The embedding method of the first-layer piggybacking framework is the same as the methods in the (13, 6) code given in this section. and the second-layer piggybacking framework adopt the diagonal-order method.
In the previous discussion, the systematic nodes in the same subset have the same repair bandwidth, and the systematic nodes in different subsets have different repair bandwidth. Suppose the systematic node i in the subset (1 ≤ ≤ L) fails, the missing data symbols need to be recovered are {f • Initially, the missing data symbols which are not embedded in piggyback blocks can be recovered via the MDS code decoding according to the structure, and the parity symbols f (1 ≤ j ≤ s−b− or s−b+1 ≤ j ≤ s), we need to obtain all the piggyback blocks P where 1≤t≤s− , and the data symbols f which is belong to the same subset with node i, where i,t ∈ , t = i. The amount of data read and downloaded in this step is (s − )ϕ .
• Consequently, the total amount of data read and downloaded to repair node i is k + (s − )ϕ . The total repair bandwidth γ sys D2 for all the k systematic nodes is Thus the average repair bandwidth ratio of systematic node, η sys D2 , is For simple analysis, let the number of systematic nodes in each subset be as equal as possible. Define the following three variables, where w and w are the quotient and remainder of k/L respectively. By Eq. (38), we have 1 1 Hence, the average repair bandwidth ratio of systematic nodes, η sys D2 , is given in (43), as shown at the bottom of the next page. A lower bound and an upper bound of η sys D2 are presented as following.
• From Eq.(43), it can be shown that η sys D2 has the smallest value when w = 0, which means that k is a multiple of L and w = w u = k L .
The right side of the inequation (44) can be a lower bound of η sys D2 for the improved double-layered piggybacking codes.
• If k is not a multiple of L, i.e., w = 0. According to the inequality of arithmetic and geometric means, we have Consequently, by Eq. (43), we obtain Hence, a upper bound of η sys D2 for the improved double-layered piggybacking codes is when (52) equals 0, L = √ 2s − 1. Then, min( low (η sys D2 )) is It's a pity that we can't obtain the min( up (η sys D2 )) exactly because (53) is a cubic equation. Clearly, min( low η sys D2 ) is a decreasing function with s and the improved framework requires s ≤ r. In order to obtain the minimum of low η sys D2 , we set s = r. Hence, Eq. (54) can be described as Eq. (55).
The curve of min( low (η sys D2 )) with different r is shown in Fig. 14.
The following are a few notes from Fig. 14, • The min( low (η sys D2 )) is dominated by the number of parity nodes r only, with no relationship to the code rate.
for L = √ 2r − 1 and p b = √ k k . At the end of this subsection, the asymptotic analyses of min( low (η all D2 )) are obtained as shown in (58), (59), and (60), as shown at the bottom of the next page. Consequently, we can get the following conclusion. VOLUME 8, 2020 • When the k tends to infinity, the min( low (η all D2 )) approaches to a constant, which is related to r.
• When the r tends to infinity, the min( low (η all D2 )) approaches to a constant, which is related to k.
• When both the k and the r tend to infinity, the min( low (η all D2 )) approaches to zero.

V. DECODING COMPLEXITY ANALYSES
In this section, we give some analyses on the decoding complexity of the proposed two double-layered piggybacking codes.

A. DECODING COMPLEXITY FOR THE FIRST DOUBLE-LAYERED PIGGYBACKING CODE 1) COMPLEXITY FOR REPAIRING A SYSTEMATIC NODE
As the statement in Section III, piggybacking codes, adopt two kinds of calculations to repair a failed systematic node, i.e., the MDS code decoding and solving piggyback functions. Recall Fig.5 in Section III. Assume that the systematic node with index i fails. We have Eq.(61), as shown at the bottom of this page. Firstly, The data symbol f are required. Distributed storage systems usually take the form of systematic codes, thus the vector representation of a s−b can be described as (f • Subtract the parity symbol from the ''black box'' to get the piggyback block. Thus, one addition is needed; • Subtract other surviving data symbols from the piggyback block. Thus, x − 1 additions are required. Actually, x is the average number of data symbols embedded in a piggyback block, i.e., x = (s−a)k (r−1)a . In this way, there are s − a missing symbols similar to the repair operation in Fig.(5). Hence, (s − a)k multiplications and (s − a)(x + k − 1) additions are cost.
In summary, sk multiplications and (s−a) 2 k (r−1)a + s(k − 1) additions are cost for repairing a systematic node.

2) COMPLEXITY FOR REPAIRING A PARITY NODE
For simplicity, recall Fig. 4 in Section III. Assume that the parity node with index k + 2 fails. First of all, f (1) i can be worked out as follows, • Compute the parity symbol f (5) k+1 in the ''black box'' where the piggyback block embedded the missing symbols in. This step costs k multiplications and k − 1 additions; • Subtract the parity symbol from the ''black box'' to get the piggyback block. Thus, one addition is needed; • Subtract other surviving parity symbols from the piggyback block. Thus, y − 1 additions are required. Actually, y is the average number of parity symbols embedded in a piggyback block, i.e., x = s−b b . In this way, there are s − b missing symbols similar to the repair operation in Fig. 5. Hence, (s − b)k multiplications and (s − b)(y + k + 1) additions are cost.
lim k→∞,r→∞ In summary, sk multiplications and s( s−b b +k −1) additions are cost to repair a parity node.

B. DECODING COMPLEXITY FOR THE IMPROVED DOUBLE-LAYERED PIGGYBACKING CODES 1) COMPLEXITY FOR REPAIRING A SYSTEMATIC NODE
For simplicity, recall Fig. 11 in Section IV. Suppose that the systematic node with index 3 fails and it is in the subset Firstly, The misssing symbol f (4) 3 is repaired via the MDS code decoding. As previously analyzed, the MDS code decoding for the recovery of f (4) 3 costs k multiplications and k − 1 additions. In this way, there are l missing symbols similar to the repair operation in Fig. 12. Hence, kl multiplications and (k − 1)l additions are cost to repair them.
Secondly, f 3 can be figured out as follows, • Compute the parity symbol f (4) k+2 . This step costs k multiplications and k − 2 additions; • Subtract the parity symbol from the ''black box'' to get the piggyback block. Thus, one addition is needed; • Subtract other surviving data symbols from the piggyback block. Thus, φ i −1 additions are required. Actually, φ i is the average number of systematic nodes in a subset, i.e., φ i = k L . In this way, there are s − l missing symbols similar to the repair operation in Fig. 12

2) COMPLEXITY FOR REPAIRING A SYSTEMATIC NODE
Because the construction of piggyback blocks designed for parity nodes remains unchanged, the complexity for repairing a parity node remains the same. sk multiplications and s( s−b b + k − 1) additions are cost to repair a parity node.

VI. COMPARISON WITH SOME EXISTING PIGGYBACKING CODES
In this section, the proposed piggybacking codes are compared with some existing piggybacking codes, including the three piggybacking schemes proposed by Rashmi et al. [24], [37], abbreviated as R-PB-1, R-PB-2, and R-PB-3 respectively, as well as the generalized piggybacking codes presented by Yuan et al. [27], called G-PB. Moreover, the first double-layered piggybacking code we proposed in the comparison is abbreviated as D-PB-1, and the second (the improved) is D-PB-2.
In Table 1, the requirements for the numbers of parity nodes and substripes for different classes of piggybacking codes are given. In this table, m is a positive integer. R-PB-2 needs more parity nodes and more substripes. For R-PB-1 and R-PB-3, the number of substripes must be even. For G-PB, the choose of parameters are more flexible while it can only repair systematic nodes. In D-PB-1, we can just add one more substripe compared with G-PB so as to repair both the systematic nodes and parity nodes. The number of substripes in D-PB-2 is no larger than r, it obtains the lowest repair bandwidth ratio.
In Fig.15, the repair bandwidth ratio for different piggybacking codes with (n, k) is shown. The piggybacking codes have the property of repair-by-transfer [38], i.e., the system only read those symbols that need to be downloaded. Thus the amount of data read is equal to the amount of data downloaded. The η sys , η par , and η all of different piggybacking codes with (n, k) are given in Fig.15(a), Fig.15(b), and Fig.15(c) respectively.
As shown in Table 2, for R-PB-1 and R-PB-3, the number of substripes is 8. For R-PB-2, it is 4(2r − 3). It is the values of k and r that influence the optimal repair bandwidth ratio for G-PB, D-PB-1, so the number of substripes can be set randomly. We set the number of substripes 10 for G-PB and D-PB-1 simply. For D-PB-2 we set the number of substripes being the same as r because of the construction. In addition, we also list the optimal substripes design parameters for the minimum repair bandwidth ratio in Table 2.
For the systematic nodes, the repair bandwidth ratio η sys D1 of D-PB-1 we proposed is equal to that of G-PB. In particular, when the code parameters n and k increase enough, the repair bandwidth ratio η sys D2 of D-PB-2 is the smallest. R-PB-3 and G-PB cannot repair the parity nodes. D-PB-1 and D-PB-2 we proposed have the smallest parity node repair bandwidth ratio. And when the code parameters n and k increase, the η all D2 for D-PB-2 decreases largely. Thus compared with other piggybacking codes, the D-PB-2 we proposed has the optimal comprehensive repair performance, see Fig.15(c).

VII. PIGGYBACKING CODES WITH MULTIPLE INSTANCES
Piggybacking codes are used in distributed storage systems, so the case that their constructions impact on the performance of storage systems cannot be ignored when the frameworks are designed. The requirement of a large number of substripes presents multiple challenges on the systems. The primary issue is that a large number of substripes results in a large number of fragmented data reading which is detrimental to the performance of disks [39]. The second issue is that numerous substripes also restrict the minimum size of files that can be handled by the code (and hence by the storage system) [37].
To the best of our knowledge, the Rotated-RS [25] codes, the (repair-optimized) EVENODD [16], [17], and the RDP codes [20], [21] are MDS, high-rate and have either a constant or linear (in k) number of substripes. However, they have fewer redundant nodes, which makes the fault-tolerant capability limited and the repair efficiency low. MSR codes are of particular interest to this paper since they are MDS. The Product-Matrix MSR codes [8] are explicit, practical MSR code constructions which have linear (in k) number of substripes. But the code rate of these codes is low. Cadambe et al. [40] show the existence of high-rate MSR codes as the number of substripes approaches infinity. The MSR constructions presented in [41] are high rate, but necessarily require the number of substripes to be exponential in k. Actually, it has been shown in [42], that an exponential number of substripes is a fundamental limitation of any high-rate MSR code optimizing both the amount of data read and downloaded [37].
The two kinds of piggybacking codes under the double-layered piggybacking framework we presented are both multiple substripes of an instance. we can achieve fewer substripes in each instance via embedding multiple instances one by one when the number of substripes is constant. Clearly, the parity symbols in the first two substripes do not participate in the repair of the systematic nodes as in Fig.4. These ''useless'' parity symbols are used to connect multiple instances. Firstly, sum up these parity symbols of b piggybacked substripes corresponding to parity nodes respectively, then add these summing symbols to the first substripe (corresponding to the parity symbols of the parity nodes respectively) of the next instance.
As shown in Fig.16, piggyback blocks for repairing parity nodes in the instance 1 are embedded into the first substripe of instance 2, and these parity messages are transmitted to the last b piggybacked substripes via encoding. Then, these parity messages will be transmitted into the instance 3 by embedding again. One can embed any number of instances in this way. Since the parity messages of the former instance are embedded into the ''useless'' substripe of the latter instance,  the repair method of the systematic nodes is unchanged. And the repair bandwidth ratio are the same, regarding the new piggyback blocks after embedding as new symbols. Since the repair method of the parity nodes are unchanged, the repair bandwidth ratio remains the same. In addition, the way of the improved double-layered piggybacking codes with multiple instances is similar.
For the requirement of node repair, each instance is independent each other before being embedded and node repair methods of every instances remain unchanged after being embedded. So the double-layered piggybacking codes with multiple instances not only have the same repair performance compared to the ones with single instance, but also have a small number of substripes, which makes its application have more practical significance in distributed storage system. Suppose that we need to get specific symbols in one piggyback block, what we need to do is that encoding, constructing and embedding these instances from left to right (former to latter) in turn until the piggyback block is created.

VIII. CONCLUSION
In this paper, initially, we propose a double-layered piggybacking framework (called D-PB-1) to repair both systematic nodes and parity nodes effectively at the same time via optimizing the proportion of the reserved substripes p a and piggybacked substripes p b . The repair bandwidth ratio for this scheme is reduced greatly. When the number of systematic nodes k and parity nodes r tends to infinity, the average repair bandwidth ratio approaches to zero. Subsequently, the improvement of the double-layered piggybacking framework (called D-PB-2) is proposed, which results in a smaller repair bandwidth ratio by changing the construction of the piggyback functions for systematic nodes. Compared with other piggybacking codes, the two classes of double-layered piggybacking codes we proposed not only have flexible requirements but also obtain the optimal comprehensive repair efficiency, especially for D-PB-2. These results not only have great practical significance to piggybacking codes but also have theoretical significance to repairing one type of the nodes and can be adopted in more other distributed storage systems.
In our future work, we will try to maintain the performance of the piggybacking scheme while making the complexity lower and promote the applications in actual systems.