Progressive Secret Image Sharing Scheme Based on Semantic Segmentation

Secret Image Sharing (SIS) plays an important role in the field of information security. At present, from the perspective of image restoration, SIS schemes are roughly divided into two categories: traditional threshold SIS schemes and Progressive SIS (PSIS) schemes. The former provides an all-or-nothing recovery mode in image reconstruction phase. If one of these shadows is damaged or lost, the reconstruction will fail. In contrast, the latter can present the original image progressively during the reconstruction phase. In this paper, a PSIS scheme based on semantic segmentation is proposed, where only specific objects or sensitive areas in the image are shared. After simple processing, the remaining background images can be publicly transmitted on insecure channels of the network. The background image will be regarded as a normal picture by people without shares. Owners who have reached the share of trapdoor numbers can recover a key target in an image. As the share increases, more key targets can be recovered. This paper is the first to propose secure secret sharing of images based on semantic targets. Through this method, it is possible to perform permission control on the target that needs to be divided and shared. Since the shared object is not the entire image, it can save storage space to a certain extent. Finally, through the analysis of the experimental results and comparison with other works, it can be concluded that the proposed scheme has a higher level of security and the amount of data that needs to be transmitted is small.


I. INTRODUCTION
With the rapid development of digital devices and cloud computing, amounts of multimedia data are uploaded to cloud storage space via the Internet. Digital images are the most common information carrier in multimedia data, and security issues have become increasingly prominent in various applications based on communication networks such as social networking and copyright transactions [1]- [4]. Many kinds of image information, such as medical images, drawings of product design, and private photos, need to be protected. In addition, the security of various types of commercial image transactions need to be ensured during distribution and sale.
Deciding how to effectively protect the integrity of digital images is an urgent issue in the field of information security, which has become an important subject in academia and industry. In order to solve various image security problems, The associate editor coordinating the review of this manuscript and approving it for publication was Zhiwu Li . image encryption, image information hiding, and SIS have become research hotspots in recent years [5]- [7]. Generally, the existing mature cryptographic algorithm will be used for image encryption processing [8]. Since encrypted image looks very different from the general images, it is easy to attract the attention of attackers. Actually, their security depends on the security of the key. The distribution and storage of key bring some difficulties to the practicability of the algorithm. Image information hiding [9] is to hide the secret image in the image carrier for transmission. The security of this technique is based on the non-disclosure of the hiding algorithm, not the security of cryptography sense. For image encryption and information hiding, if the encrypted image or embedded image is damaged or lost, the original image cannot be recovered.
SIS schemes [10]- [12] can solve the problems mentioned above. In SIS, an image to be protected can be divided into multiple shares that are stored separately by different participants. The secret image can be reconstructed if the number of VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ shares reaches the threshold k. Even when parts of the shares are destroyed or lost, the remaining participants could still complete the work of image restoration. Furthermore, when the confidential images are stored in the carrier image by secret sharing, they can avoid the attention of attackers, which further enhances the security during storage and transmission. There are mainly two types of SIS: Visual Cryptography (VC) [13], [14], and polynomial interpolation based SIS schemes [15], [16]. The human visual system is used in the first type of schemes and there are no complicated calculations used during the recovering phase. In fact, no complex cryptographic theories are used in these schemes. An obvious disadvantage is the large pixel expansion. Therefore, pixel values will be lost and the original image cannot be exactly recovered if SIS based on VC is used. The second type of schemes are mainly based on Shamir's threshold secret sharing scheme [10], in which polynomial interpolation is used to share and accurately recover the secret. They have been widely used in various image sharing fields [17], [18].
The threshold-based secret sharing scheme was first proposed by Shamir [10]. When using (k, n)-threshold in cryptography, a secret is divided into n shares. Then any k(k < n) shares can recover the secret without loss, and less than k shares get nothing on the secret. Thein and Lin [17] propose an SIS scheme, which is the first scheme to use secret sharing technology for images. In their work, they divide the pixels in the image into multiple sections; each section has k pixels and each pixel can only belong to a specific section. Then let the pixels of each section be the coefficients of the Lagrange interpolation polynomial, respectively. Finally, n shadow images are generated from the secret image. The scheme can reduce the size of each shadow image to 1 r of that of the secret image's.
In recent years, more threshold-based SIS schemes have been proposed [19], [20]. All of these (k, n) SIS schemes meet two features: (1) less than k shadows reveal no image information, (2) any k shadows can get the entire secret image. That is, this type of schemes provides an all-ornothing recovery mode. However, such schemes have certain limitations. For example, there are n−k shadows that will cause redundancy and will result in a waste of storage space. Besides, the greater the number of shadows, the greater the number of participants, which will increase the cost. In practice, for some images, such as medical and military images, we only care about specific objects in the image rather than the entire image. These objects must be restored according to the rights of share owners. In this case, the above SIS schemes are not applicable.
In order to meet the corresponding application requirements and improve the flexibility of the scheme, PSIS schemes have also been widely studied [21]- [25]. Compared with the traditional SIS schemes, (k, n) PSIS schemes can gradually recover the secret image and meet three features: (1) Less than k shadows reveal no image information. (2) Any k shadows can restore part of the image. (3) After reaching k shadows, each additional shadow can restore a new part of the content. Only when n shadows participate in the restoration of the image, can the secret image be completely restored.
The study in [26] proposes a random-grid-based progressive visual secret sharing scheme and the priority weighting of each share can be adjusted. However, the scheme has pixel expansion, which makes the recovered image visually different from the original secret image. The approach in [27] develops a scalable SIS scheme. In their work, there are three modes for sharing secret image, and two image division methods are used: one is based on the spatial domain to divide the image into multiple non-overlapping sub-images, and the other is based on the pixel domain. The work in [28] dwells on polynomial based PSIS with a smaller shadow size that is a little larger than |O| n and less than |O| k . The secret image can be reconstructed more smoothly from k to n shadows and the scheme is also based on the spatial domain when dividing secret images. The study in [29] develops two (k, n) scalable SIS schemes to satisfy the dynamical security requirement, and the scheme is also based on the spatial domain to divide the image. All PSIS schemes can gradually recover the secret image.
At present, all of these schemes are based either on the spatial domain or pixel domain to divide the secret image. The shared object is the entire image, and the sub-images are not assigned priority based on their importance. For some applications, such as some images in the military, only the sensitive areas related to privacy or national interests in the image, or specific objects, are the objects that we need to protect. They are also the objects we need to share. As for the rest of the image, after certain processing, it can be transmitted publicly on the internet. Therefore, these PSIS schemes are not applicable.
In this paper, we mainly focus on the rights allocation and privacy protection of digital images when sharing. Based on instance aware semantic segmentation in computer vision problems, independent objects in images are used as research objects, and a new (k, n) PSIS scheme based on semantic segmentation is proposed. This scheme has a strong purpose in protecting and restoring the original image. By segmenting the image based on semantics, the separated target objects are used as sub-images. Each sub-image is assigned priority according to its importance. The advantage of this scheme is that it can save storage space and other resources to some extent. In addition, relevant persons can gradually restore specific target objects according to their needs in specific application scenarios. Actually, it also improves some of the deficiencies in the existing PSIS schemes and improves the flexibility. Therefore, it has strong practicality.
The rest of this paper is organized as follows: In next section, we prepare some preliminaries, which include Thien-Lin secret sharing scheme [17], Guo-Ma PSIS scheme [28] and image semantic segmentation [30]. In Section 3, we propose a new (k, n) PSIS scheme based on semantic segmentation. In Section 4, experimental results and analyses are used to show the performance and superiority of the proposed scheme. Comparisons with related works are given in Section 5. The conclusion is included in Section 6.

II. PRELIMINARIES
Two representative SIS schemes, namely Thien-Lin (k, n) SIS scheme [17] and Guo-Ma PSIS scheme [28], and image semantic segmentation are introduced in this section. In reality, a lot of malleable schemes have been built. We will briefly describe Guo-Ma PSIS scheme and discuss the proposed scheme in detail.
In [17], Thien and Lin propose an SIS scheme, in which Lagrange-based interpolation polynomial is constructed, and the pixels of secret image are the coefficients of the polynomial. The scheme is divided into two phases: sharing phase and reveal phase. The details are shown as below.
Sharing phase: is the total sections of the original secret image, and p Let r pixels in each section o i be the r coefficients of the polynomial, respectively: Recovering phase: First, randomly select r shadow images and perform the following two steps.
1) Take the first element of each shadow image, and then construct a Lagrange interpolation polynomial to solve for the r coefficients, the corresponding r values of section o 1 . 2) Repeat the first step until all pixels of r shadow images are processed. When the values of each section o i are obtained, the secret image can be recovered.
The study in [28] proposes a new (k, n) PSIS scheme based on polynomial, which can reduce the shadow size. The scheme is divided into two phases: shadow generation phase and image restoration phase. The details are described in the following.
Image recover phase: Suppose that t(k ≤ t ≤ n) shadows P 1 , P 2 , · · · , P t are available. The recovery process is as follows.
Thien-Lin's scheme. If each sub-image I i is recovered, then the orginal image I can be constructed.

C. SEMANTIC-BASED IMAGE SEGMENTATION
Semantic segmentation [30] is a very important field in computer vision. It refers to identifying images at the pixel level, that is, marking the object category to which each pixel in the image belongs. The following figure is an example of semantic segmentation, whose goal is to predict the class label of each pixel in the image, see Fig. 1. In fact, semantic segmentation is to artificially segment the area in the image that expresses a specific meaning, which is based on the content of the image, namely the object perceived by the human visual system. The separated parts are meaningful content, representing different categories of objects. In addition, the segmentation methods are based on contour segmentation.
In this paper, the secret image is segmented based on semantics. It means that the region or target objects in the image with specific meaning is segmented. The segmentation method adopted is not based on the contour, but the rectangular area where the target objects in the image are located is segmented out, see Fig. 2. In our proposed scheme, the idea of image semantics is used to construct a new PSIS scheme.  The focus of the research is not the semantic segmentation method, but the adoption of this idea to generate sub-images, which are the secret shared objects.
In short, we hoped that more scholars can use machine learning and deep learning to do semantic segmentation. Then, the separated objects are used as the objects of secret sharing. We also hope that increasing researchers will deeply integrate image semantic segmentation with SIS technology to enrich SIS schemes.

III. THE PROPOSED SCHEME A. MOTIVATION
At present, since images are widely used in many fields, it has become very important to ensure that images can be safely stored and transmitted on the Internet. In addition, the traditional encryption technology and image information hiding technology cannot fully ensure the secure storage and transmission of images. In comparison, SIS is more conducive to the secure storage and transmission of images.
As the research progresses, a large number of SIS schemes have been proposed. In general, these schemes are divided into two categories: PSIS and non-progressive SIS. A common feature of these two types of schemes is that the original image is divided into multiple non-overlapping sub-images based on spatial or pixel domain of the image, and then the secret sharing technology is used to generate multiple shares and distribute them to multiple participants. One obvious disadvantage of these schemes is that the corresponding sub-images are restored randomly according to the obtained shadow images, and the recovered content has a lot of randomness. Besides, the recovered part is less readable.
Generally, in most cases, we care about only one or some target objects in an image, and the rest target objects and background are not important. For example, in telemedicine diagnostic system, for some medical images, only the location of the lesion, or the sensitive area needs to be protected. The remaining parts that do not involve patients privacy can be publicly transmitted. Besides, some pedestrian detection and target tracking focus only on specific targets; therefore, such objects are protected objects.
In this paper, a new (k, n) PSIS scheme based on semantic segmentation is proposed. Since the specific objects or sensitive areas in an image are protected, this scheme has a strong purpose in protecting and restoring the original image. Therefore, it can save storage space and other resources to some extent.

B. THE PROPOSED (K , N) PSIS SCHEME
The proposed scheme includes two phases: a sharing phase and a reconstruction phase. In the sharing phase, shadow generation is similar to Liu-Yang's scheme [29], and the reconstruction of orginal image is the same as in Thien-Lin's scheme [17]. The steps are shown as below.
Sharing phase: 1) First, convert the original secret image to grayscale. Then, we identify sensitive areas of the original image. Several original target sub-images are separated based on semantics from the original secret image O with size m × n, marked as P 1 , P 2 , P 3 , · · · , P t , where t is the number of the objects that need to be protected in O. At the same time, we record the position information of each sub-image (such as the coordinate information). After that, fill the rest and represent it by B. In this way, it will not be suspected when transmitting online. 2) Determine the number of pixels in x-axis and y-axis directions of all target sub-images, represented by M 1 , M 2 , · · · , M t and N 1 , N 2 , · · · , N t , respectively.
The value of i is inversely proportional to the priority of its corresponding sub-image, i.e., the smaller the value of i , the higher the priority of the corresponding sub-image. 4) For PN i , i ∈ [1, t], use (k + i − 1, n) threshold scheme to generate n shares s i,1 , s i,2 , · · · , s i,n . 5) Generate n shadows S j , j = 1, 2, 3, · · · , n, S j = j s q,j , j ∈ [1, n], q ∈ [1, m], and then distribute them to n different participants.
Reconstruction of the secret image: 1) First, take l shadows S 1 , S 2 , · · · , S l , when l ≥ k, and then we can restore the original image. If l = k, we can recover PN 1 with the highest priority by constructing Lagrange interpolation polynomial. If we add another shadow, then PN 2 can be recovered. When all shadows are available, all sub-images can be reconstructed. 2) Use image processing technology to obtain P i , i ∈ [1, t], and then reconstruct the original image based on the original position information of P i in the image. Finally, O = P i ∪ B. The secret image is reconstructed completely.

IV. EXPERIMENTAL RESULTS AND ANALYSIS A. EXPERIMENTAL RESULTS
In this section, we will give two examples to verify the performance of the proposed scheme. Next, we will analyze the first example in detail. Example 1: The original secret image O, processed background B and two target sub-images segmented from O are shown in Fig. 3. First, semantic segmentation is used to obtain the original target sub-images P 1 , P 2 . Then, resize P 1 , P 2 to obtain the two new sub-images represented by PN 1 , PN 2 , which are shown in (c) and (d) in Fig. 3. Suppose that these two are sensitive areas, the objects that need to be protected in an application scenario. Here, we assume that PN 1 has a higher priority than PN 2 . Let PN 1 = PN 1 , PN 2 = PN 2 . Then, fill the remaining part of the original secret image to obtain the background image B. In the proposed scheme, let k = 5, n = 8. Next, n shadows need to be generated, see Fig. 4, and the specific steps are as follows. Generation of shadows: First, for PN 1 , we need to construct a k − 1 degree polynomial: where j represents the sub-image subscript, and j = 1, 2; i represents the specific block in the sub-image; (i, 0), (i, 1), · · · , (i, k −1) represent the elements of a specific block in a sub-image. The generation of polynomial is the same as in Thien-Lin's scheme. Here, for PN 1 , we find five polynomials: then we can obtain multiple shares: Next, for PN 2 , a similar approach is used to generate multiple shares. The only difference is k = 6. We construct five polynomials in the same way: then, we can obtain multiple shares: s 1 2,1 = f 1 2 (9), s 1 2,2 = f 1 2 (10), · · · , s 1 2,8 = f 1 2 (16) s 2 2,1 = f 2 2 (9), s 2 2,2 = f 2 2 (10), · · · , s 2 2,8 = f 2 2 (16) s 3 2,1 = f 3 2 (9), s 3 2,2 = f 3 2 (10), · · · , s 3 2,8 = f 3 2 (16) s 4 2,1 = f 4 2 (9), s 4 2,2 = f 4 2 (10), · · · , s 4 2,8 = f 4 2 (16) s 5 2,1 = f 5 2 (9), s 5 2,2 = f 5 2 (10), · · · , s 5 2,8 = f 5 2 (16) Note that in order to make the obtained 8 shadow images more distinguishable, x = 9, 10, · · · , 16, respectively. Finally, we can generate eight shadows, see Fig. 4. All shadows are shown in Fig. 4. Then, these shadows will be distributed to different participants. Reconstruction of the secret image: In the image reconstruction phase, only when the number of shadows reaches the threshold, can the restoration of the original image be performed. Otherwise, the restoration fails. In addition, since PN 1 has a higher priority than PN 2 , when we obtain five shadows, we can recover the sub-image PN 1 by constructing Lagrange interpolation polynomial, see Fig. 5(a). PN 2 can only be restored when there are six or more shadows. Finally, PN 1 and PN 2 are processed to obtain the original sub-images P 1 and P 2 . Then, P 1 and P 2 are placed in the original position in the secret image. Finally, the original secret image O can be reconstructed losslessly, as shown in Fig. 5(b). Actually, the recovery of sub-images is similar to Thien-Lin's scheme. Example 2: Similar to in the first example, the secret image needs to be segmented to generate sub-images. Then, shadow images are generated by the sub-images. Finally, the secret image is restored.
The image I , processed background B and two target sub-images segmented from I are shown in Fig. 6. First, semantic segmentation is used to obtain the original target sub-images P 1 , P 2 . Then, resize P 1 , P 2 to obtain the two new sub-images represented by PN 1 , PN 2 , which are shown in (c) and (d) in Fig. 6. Suppose that these two are sensitive areas, the objects that need to be protected. Here, we assume that PN 1 has a higher priority than PN 2 . Let PN 1 = PN 1 , PN 2 = PN 2 . Then, fill the remaining part of the original secret image to obtain the background image B. Here, let k = 3, n = 5. Next, n shadows need to be generated, see Fig. 7, and the specific steps are as follows.

B. PERFORMANCE AND SECURITY ANALYSIS
A novel (k, n) PSIS scheme based on semantic segmentation is proposed in this paper. Secret image segmentation is performed based on semantics. Note that the proposed scheme is suitable for images with two or more non-overlapping sensitive areas or target objects. Only specific objects in the image participate in the generation of shadows. The rest is filled as a background image and can be publicly transmitted or restored on the network. In addition, since the background image is filled, it will not cause the attackers' suspicion. Furthermore, this solution sets the priority for the divided different objects. The higher the priority of the target object, the less shadows are required during the recovery phase.
In practice, our proposed scheme has some advantages. First, the proposed scheme is the first to use image semantic segmentation for PSIS, which provides a new research idea for other researchers in the study of SIS schemes. Second, the advantage of this image segmentation method is that it saves storage space, since the secret sharing technology does not target the entire image, but a specific area in the image. Therefore, this method reduces the shadow size. Whether it is image storage or transmission, it will greatly save resources and bring convenience to image information hiding. Finally, since the background image is publicly transmitted on the Internet, if it is lost or damaged during the transmission process, we can request the sender to send it again. Thus, the loss of the background image will not affect the recovery of the final secret image. Furthermore, if only part of the shares is lost and the number of shares available can still reach the recovery threshold, the final secret image can still be recovered.
In addition, the proposed (k, n) PSIS scheme meets security requirements. On the one hand, it is determined by the characteristics of secret sharing technology; on the other hand, semantic segmentation method is used to hide important information of the image. Thus, the original secret image is presented in a completely new way, which can avoid attracting people's attention. The secret sharing method used is similar to that of Liu-Yang's scheme [29]. Based on that, our scheme sets the priority for the divided sub-images. In practical applications, a sub-image can be recovered based on the needs of relevant parties.
First, we will discuss whether the division of sub-images has practical significance. In SIS schemes, before the shadow is generated, the secret image is often divided into multiple sub-images. In this paper, we divide the original image according to the content of the secret image. This means the secret image is divided based on image semantics. The sensitive areas or the specific objects in the image are separated, and they are the shared objects. The advantage of this technique is that relevant persons can gradually restore the secret image according to the application requirements in certain fields. At present, the existing SIS schemes such as those in [13], [15], [28], [29] have not yet segmented images based on semantics, the entire secret image is usually divided into multiple non-overlapping sub-images based on the spatial domain.
Next, this paper sets priority to the sub-images. In our solution, since the image segmentation method based on semantics is used, the separated sub-images have different practical significances. Then we assign priorities to these sub-images based on the semantic content of the sub-images. The higher the priority is, the fewer shadows are needed during the recovery phase. In contrast, there is no priority involved in these schemes such as those in [15], [24], [28], [29], [31].
Finally, we will compare the size of the shadow image. In the proposed scheme, we do not secretly share the entire image. Instead, we let one or more specific target objects VOLUME 8, 2020 in the image be the secretly shared objects. The shadow size in the proposed scheme is n|M × N | t−1 i=0 1 k+i , where |M × N | and t are the size and number of the separated sub-images, respectively. Therefore, n|M × N | t−1 i=0 1 k+i < n|O| t−1 i=0 1 k+i , where |O| is the size of the secret image. In [15], the shadow size is |O| k , where k < k. It is obvious that the shadow size in [15] depends on dynamic threshold k . The shadow size in [28] is (k+1)|O| k(n+1) , which depends on the values of k and n. It is easy to prove that (k+1)|O| k(n+1) < |O| k . However, in our scheme, the shadow size depends on the values of k and n and the size and number of the separated sub-images. A novel (n, n) SIS scheme in [31] is proposed. The shadow size is |O|, which is larger than that in [15]. In fact, if the target sub-images are very small, the shadow size in the proposed scheme will be very small. Therefore, in our scheme, the specific shadow size depends on the characteristics of the secret image in practical applications.
Actually, in the existing SIS schemes, the shadow size and smoothness when restoring the original image are commonly used indicators to measure the performance of the schemes. The sub-images in the proposed scheme are obtained based on semantic segmentation and the shared object is not the entire secret image, but the separated sub-images. Therefore, the smoothness of the recovery depends on the number of separated sub-images. This is different from other PSIS schemes like [28]. Besides, the five differences including the threshold property, segmentation method, shared object and progressive between the proposed scheme and related schemes are given in Table 1. In Table 1, the schemes in [15], [22], [28], [29] and [31] are different; k, k , k are different thresholds. In this paper, the threshold is not limited by the scheme itself, and it just satisfies the condition of k < n. In addition, the unique feature of the proposed scheme is to segment the original secret image based on semantics and let the divided sub-images which are just a part of the secret image as the shared objects.

VI. CONCLUSION
At present, image semantic segmentation technology is widely used in many fields, e.g., automatic driving of cars, diagnosis of medical images etc. When these divided images are stored and transmitted on the network, it is important to ensure their security.
In this paper, a new (k, n) PSIS scheme based on semantic segmentation is proposed. First, according to the characteristics of the secret image and the needs of relevant users or specific application scenarios, the confidential image is segmented based on semantics. Only particular objects or sensitive areas in the image will be used for secret sharing, and the remaining part can be publicly transmitted or stored after being processed. In addition, the segmented target sub-images are assigned different priorities. The higher the priority, the less shadows are needed to restore it, and by analogy, the sub-image with the lowest priority will be the last one to be restored. This solution not only reduces the shadow size, saves storage space, but also provides convenience for hiding image information. In fact, the background image in the proposed scheme is processed by filling, and the resulting image is still insufficient. Therefore, our future work is to find the optimal image processing method including selecting a more appropriate image semantic segmentation method. It aims to make the processed image look natural, thus, it will not be suspected by the attackers. We can also apply the proposed scheme to social networks [32], multi-agent systems [33], automated manufacturing systems [34] and cyber physical systems [35].
LINA ZHANG received the Ph.D. degree in applied mathematics from Wuhan University, in 2008. She is currently an Associate Professor with the College of Computer Science and Technology, Xi'an University of Science and Technology. Her research interests include cryptography and information security.
XIANGQIN ZHENG received the B.S. degree in the Internet of Things from Suzhou University. She is currently pursuing the master's degree with the College of Computer Science and Technology, Xi'an University of Science and Technology. Her research interests include cryptography and information security.
YING LIU is currently a Senior Engineer with the National Key Laboratory for Complex Systems Simulation, Beijing, China. Her research interests include information security, artificial intelligence, and intelligent system security and control. VOLUME 8, 2020