A Privacy-Preserving Game Model for Local Differential Privacy by Using Information-Theoretic Approach

Local differential privacy (LDP) is an effective privacy-preserving model to address the problems which do not have a trusted entity. The main idea of the LDP is to add randomness in real data to guarantee individual’s private sensitive information. Here, the technology of randomized response is an effective method to realize the LDP mechanism. In fact, the randomized response is a probabilistic mapping from the real data to perturbed data, which can be modeled as an information-theoretic lossy compression mechanism. What’s more, the privacy budget <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> has become a <italic>de facto</italic> standard to quantify the worst-case privacy leakage. However, such a metrics can not capture the question that which one is the optimal privacy mechanism in a set of equivalent <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula>-privacy mechanisms. Besides, the privacy and utility are closely correlated with the privacy mechanism, and existing methods do not consider the strategic adversary’s behavior. In this paper, we tackle the problem of tradeoffs privacy and utility under the rational framework within an information-theoretic approach as the metrics. To address the problem, we first formulate this trade-off as a minimax information leakage problem. Then, we propose a privacy preserving attack and defense (PPAD) game framework, that is, a two-person zero-sum (TPZS) game. Further, we develop an alternating optimization algorithm to compute the saddle point of the proposed PPAD game. As a case study, we apply our method to compare several alternative <inline-formula> <tex-math notation="LaTeX">$\ln 2$ </tex-math></inline-formula>-privacy mechanisms, the experimental result demonstrates that can provide an effective method to compare equivalent <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula>-privacy mechanisms. Furthermore, the numeric simulation result confirms that the proposed method also be useful for the protector to assess privacy disclosure risks.


I. INTRODUCTION
The problem of leaking private sensitive information is widely concerned with society and academia, and is becoming one of the main challenges in today's big data era. The privacy issues bring the demands of privacy protection for data collection, data release as well as data analysis, which urgently need the effective privacy protection models and The associate editor coordinating the review of this manuscript and approving it for publication was Muhammad Asif . algorithms. Specifically, the differential privacy (DP) [1], [2] is a privacy protocol, which provides rigorous data privacy guarantees. In fact, the DP has become a de facto standard for privacy-preserving community because it has a rigorous mathematical proof of privacy guarantees. In general, DP is classified as two working settings, i.e., centralized setting and local setting. In the centralized setting [2], a trusted data curator performs the privacy protocol to protect sensitive data records. The basic idea is to add randomness to the accurate results. However, there is not always having the VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ trustable data curator who processes the data. To this end, local differential privacy [3] (LDP) is proposed to address the problem of untrustworthy data curator. In the LDP, each user perturbs her real data locally before sending reported data to the data aggregator. The original motivation of the LDP is achieved by randomized response, which was first discussed in [4]. In recent years, many state-of-the-art LDP mechanisms have been developed (e.g., randomized aggregatable privacy-preserving ordinal response, RAPPOR [5], [6], k-ary randomized response, k-RR [7]) to provide privacy preserving. Currently, the LDP is widely investigated in the privacy protection area, and has been applied into privacy-preserving data collecting and analyzing, such as Google Chrome browser [5], Apple's OS [8] etc. Specifically, each user independently sends her own reported data to the aggregator in the data collection scenario. Afterwards, the aggregator collects, stores these reported data, and then prefers to infer the individual's information from the collected data since he may be an honest-but-curious adversary. In such case, each user performs LDP locally to protect her privacy. In practice, the randomized response (RR) technique is an effective method to achieve LDP [7], [9]- [11]. Essentially the RR mechanism is a probabilistic mapping from real data to disguised data. Thus the randomness of privacy-preserving mechanism corresponds to the problem of trade-off between privacy protection and data accuracy, which is the well-known problem of privacy-utility tradeoff. At present, this problem is still the concern of academic research.
Indeed, the LDP mechanism can be modeled as a noisy channel from the perspective of private information flow [12], [13]. Meanwhile, privacy and utility metrics are the fundamental work to investigate the privacy-utility tradeoff. Currently, the privacy budget is a de facto standard to quantify the indistinguishability level. However, it still has its drawbacks. As mentioned in [14], a deterministic privacy protocol Q(x) = x mod 2 which provides privacy guarantee with is infinity, but it still prevents some privacy disclosure. In addition, such a metrics can not evaluate equivalent -indistinguishability mechanisms. To solve these drawbacks, information-theoretic approach is used to measure privacy leakage, and has been widely studied in recent years [7], [15]- [18]. The mutual information (MI) measures how much information about real data is contained in disguised data. It captures the aggregators' knowledge, and assumes aggregator does not know the true distribution exactly, but only knows it lies in a probability distribution set [7], [13]. The MI has its advantage to solve this problem. In fact, the users aim to reduce privacy leakage, they are analogous to privacy defender. However, the goal of an aggregator is to obtain the privacy statistics information and attempts to infer personal information, who is similar to privacy attacker. Thus, the concerned problem evolves into a privacy attack-defense game between attacker and defender.
Based on those analyses mentioned above, the objective of this paper is to seek an optimal privacy mechanism by analyzing the actions of privacy attacker and defender. Intuitively, the more randomness of the privacy mechanism will be obtained the better privacy performance. The works [16], [18], [19] utilize MI measuring the privacy leakage about privacy-preserving mechanisms because the notion of MI has a clear meaning, that is, measuring the amount of uncertainty reduction about original information. However, the works [7], [20] adopt MI as utility metrics, and further preserve the useful information as much as possible. Inspired by these works, the problem mentioned above would become a minimax problem, and naturally evolved into a two-person zero-sum game. In this paper, we consider formalizing the concerned problem as an attack and defense game about privacy since the game theory has its advantages to deal with such a problem. Then, we provide the analytic results, which confirm that our method can be used for privacy defender to take optimal privacy strategy.

A. OUR CONTRIBUTION
The major contributions of our work can be summarized as follows: 1) To analyze the rational actions between users and aggregator, we propose a general game-theoretic framework of privacy attack and defense, PPAD, and quantify the private information gain of aggregator based on the information-theoretic approach. 2) We formalize the objectives of privacy defender and attacker as a minimax MI privacy problem, and then construct a two-person zero-sum game to solve the formalized minimax problem. 3) We propose an effective method to evaluate equivalent -privacy mechanisms. Further, we demonstrate the MI privacy leakage can reach the upper bound under the worst-case, and that can be used to assess privacy disclosure risks.

B. PAPER OUTLINE
The remainder of this paper is organized as follows. Section II reviews the related work about our research topic, and Section III introduces the preliminaries to the paper. Section IV presents the system model and problem formalization. Section V describes the details of privacy attack and defense game model, and presents its theoretic analysis. A case study and numerical simulation results are given in Section VI, followed by the conclusion and future work in Section VII.

II. RELATED WORK
Differential privacy (DP), has been widely studied in a series of state-of-the-art literatures (e.g., [1], [2], [21], [22]). In recent years, many researchers began to measure privacy leakage and usability with information-theoretic approach (e.g., [13], [17], [23]), which is used to investigate the optimal DP mechanism. What's more, utility-privacy tradeoffs has been became an important problem in the privacy-preserving community. In addressing this problem, the game theoretic idea is used to study it under the DP becoming a fascinating research topic. In the following, we survey the important works in each aspect. Firstly, the privacy-preserving mechanism can be modeled as a noisy channel model [13], [17], [23], [24], including the radio channel model [25] connected with IoT devices. Then, the privacy leakage can be measured by entropy [19], [26], [27]. In particular, Alvim et al. proposed measure information uncertainty based on the idea of quantitative information flow (QIF). Moreover, the notion of mutual information has also been considered in [15], [18], [28], and given a formal definition of MI-privacy. Wang et al. [18] formulated a MI-privacy optimization program, and proved that the MI optimal mechanism guarantees a certain level of differential privacy. In addition, Cuff et al. [28] adopted MI given an equivalent definition of differential privacy. The works of [15] and [29] given a fundamental relation between MI and differential privacy.
Secondly, the information-theoretic approach has also been used in the LDP setting. It is indeed true that the LDP can be represented by a probabilistic mapping from original data domain X to a regenerated space Y . Let Q(y|x) be the conditional probability of input x ∈ X and output y ∈ Y . As such, the LDP mechanism is captured by a conditional probability Q (s.t. y Q(·|x) = 1 and Q(·|x) ≥ 0). In this line of research, [7] proposed k-RR mechanism by using MI as utility metrics. Also, [7], [30] proposed k-RR mechanism is optimal with the probability Q(y|x) = e |X |+e −1 for all x = y, and Q(y|x) = 1 |X |+e −1 for x = y. As such, the LDP established a fundamental relation with information-theoretic noisy channel model, and also represented by a probability transfer matrix. Based on this fundamental relation, [14] presented a new metrics approach for LDP from information-theoretic perspective. Besides, [13], [17] investigated the optimal LDP mechanism using information-theoretic approach. In summary, the information-theoretic approach which applied into the LDP has attracted lots of attentions.
Finally, game theory as an effective analysis tool for the issue that existing conflict and competition, has been widely studied in data security recent years (e.g., [31]- [33]). In the application of differential privacy, Xiao et al. [34] formulated a privacy aware recommendation game to evaluate the performance of the proposed deep reinforcement learning based user profile perturbation scheme, which applies differential privacy to protect user privacy within recommendation services. Besides, the well known two-player zerosum game model has been considered by [35]- [37]. What's more, [36]- [38] considered information leakage game model, which are constructed from the perspective of QIF. Additionally, the non-cooperative differential game [39] and the Stackelberg game [40] have been studied in privacy-preserving under differential privacy. From these related work about the game and privacy preserving, we can conclude that the problem of utility-privacy trade-off solved by game-theoretic approach is becoming an effective method.

III. PRELIMINARIES
In this section, some basics will be summarized for our usage in this paper for readers' convenience. Here, we only give a brief introduction because of space limitation, if readers need, refer to the relevant materials for more details.

A. LOCAL DIFFERENTIAL PRIVACY SETTING
Let X be a discrete random variable, which takes its value from a candidate set X , and x ∈ X represents user's private data. In order to protect the secret data, a randomized privacy-preserving mechanism will be used to produce a disguised data. We assume the disguised data is a discrete random variable Y , and its value y comes from a candidate set Y . Thus the privacy-preserving mechanism forms a probabilistic mapping from X to Y . In fact, DP has an underlying assumption, i.e., there is a trusted data curator who has the real data. However, it is generally a semi-honest participator who follows the privacy protocol, but attempts to obtain private information. In such case, each user locally perturbs her original data to obtain disguised data, and then sends it to the aggregator. As such, the randomized mechanism is an uncertain function mapping Q(y|x). Thus, we give the following equivalent definition.
Definition 1 (Local differential privacy, LDP): Let X and Y are finite discrete sets, a probabilistic function Q mapping X to Y , denoted as Q(y|x) y∈Y ,x∈X : X → Y . It is a -LDP mechanism, if and only if it satisfies for all x, x ∈ X and y ∈ Y . Further, the privacy budget is a positive real number, which is defined as In particular, the privacy budget describes the strength of privacy preserving as well as provides a quantitative method to measure privacy leakage. A natural question is: how does the privacy budget affect the privacy leakage? Intuitively, the is a metrics, which measure the probabilistic distinguishability level of obtaining same output for any two distinct inputs. Further, a smaller demonstrates a greater indistinguishability, that is, the attacker can hardly completely identify x. Thus leading to less privacy leakage.
Remark 1: The definition 1 shows that LDP provides the worst-case privacy guarantee, and the privacy budget is independent with source probability distribution P(x) but only depends on the conditional probability Q(y|x).

B. INFORMATION-THEORETIC METRICS
Let X and Y are discrete random variables, and X ∈ X , Y ∈ Y . Therefore, a probabilistic function Q maps X to Y forming a typical Shannon [41] discrete noise channel Q : X × Y → R, which is represented by a probability matrix Q(y|x) (s.t. 0 ≤ Q(y|x) ≤ 1 and y∈Y Q(y|x) = 1 for all y ∈ Y and x ∈ X ). VOLUME 8, 2020 The notion of entropy is proposed to measure the uncertainty about random variable X . In particular, Shannon entropy [41], [42] is a popular metrics method, and which is defined as H (X ) = − x∈X P(x) log P(x). Then, the notion of conditional entropy defines the remaining uncertainty about X after observing Y , denoted as H (X |Y ) = E y∈Y [H (X |Y = y)]. Further, the mutual information I (X ; Y ) measures the amount of uncertainty reduction about X , i.e., the amount of information has learned from X by knowing Y , denoted as I (X ; Y ) = H (X ) − H (X |Y ). More specifically, I (X ; Y ) quantifies how much information flow from X to Y , that is the basic idea of QIF [24]. Therefore, MI has an important property, that is, it will always be nonnegative. Furthermore, the relation between entropy and MI indicates that it can be calculated by where In addition, MI as an information measure method, it considers the influences of Q(y|x) as well as the prior distribution P(x). That is to say, it reflects the statistical characteristics of the specific noisy channel.

C. TWO-PERSON ZERO-SUM GAME AND MINIMAX THEOREM
Next, we consider a two-person attack and defense game, that is, a game between attacker and defender. Formally, such a game is defined as (D, A , u D , u A ), of which D, A be nonempty finite sets, and represent defender's and attacker's available actions, respectively. Furthermore, the measure functions u D : D × A → R and u A : D × A → R map the Cartesian product of D and A to a real number. More specifically, they are payoff functions of defender and attacker, respectively.
In addition, the players of a game always be assumed as rational decision makers, and they pursue to maximize their payoff functions. In particular, for any s d ∈ D and s a ∈ A , the sum of payoff function u d (s d , s a ) and u a (s d , s a ) equals to zero, i.e., the defender's loss is equivalent to the attacker's gain. Thus, the goal of attacker is to maximize the payoff function, while the defender is to minimize it. Usually, the two-person zero-sum game always corresponds to the minimax problem. As for this situation, the well-known von Neumann's minimax theorem [43] provides an effective analysis method. In the following, we give a brief introduction.
Theorem 1 (von Neumann's minimax theorem): Let P and Q be nonempty compact, convex subsets of Euclidean space, and U : P × Q → R be a continuous function. If U (P, Q) is quasiconcave for all P ∈ P and quasiconvex for all Q ∈ Q. Then it has The notion of saddle point is related to minimax Theorem 1, that is to say, if P * ∈ P, Q * ∈ Q be the saddle point of U (P, Q), when they satisfy for all P ∈ P and Q ∈ Q. This is equivalent to say, Then, (P * , Q * ) is called as the saddle point of function U (P, Q) in P × Q.

IV. PROBLEM STATEMENT
In this section, we introduce the data collection architecture of this paper, and then establish its system model. Further, we formally define our research problem.

A. APPLICATION ARCHITECTURE
Our application architecture is depicted in Figure 1, where a number of users and an aggregator participate in the data process procedures. To protect personal privacy, each user performs the privacy-preserving protocol to perturb her own secret data. In fact, an informed aggregator may know something a priori about the data distribution. To capture this priori, we assume he only knows the distribution lies in a set but does not know it exactly. In this situation, users and aggregator are considered as rational players, and they decide the privacy mechanism together. When the protocol has to be agreed, the process of the privacy-preserving data collection follows three steps. Step 1: The aggregator releases a signal of data collection task, and determines the details of data to be collected. The data to be collected might include individual data such as home address, marital status. Then, the aggregator recruits users to report their own data.
Step 2: We consider the rational decision makers that they can decide whether or not they will report their data to the aggregator. If a user agrees to participate the current collection task, she performs the privacy protocol to derive reported data, and then sends it to the aggregator.
Step 3: Based on steps 1 and 2, the data aggregator collects, stores users' reported data, and then analyzes these collected data.
Problem: We consider an honest-but-curious aggregator who follows the privacy-preserving protocol but desires to infer personal information from the reported data. Thus our objective is to seek an optimal privacy mechanism to trade-off privacy and usability, that is, an optimization mechanism both for user and aggregator.

B. SYSTEM MODEL
We consider there are n users in the privacy-preserving data collection system, [n] = {1, 2, · · · , n}, X (resp. Y ) be a finite set that represents all possible values of personal data (resp. disguised data). Let X (resp. Y ) be a discrete random variable representing an individual's data (resp. disguised data). Furthermore, let |X | be the number of distinct atoms in X , and use a set of integers increase from 1 to |X | to represent the real ordinal number in X . The LDP forms a probabilistic function which maps x ∈ X to y ∈ Y with probability Q(y|x), denoted as Q : X → Y . In addition, we use the subscript x i (resp. y i ) to represent the personal data (resp. disguised data) of ith user in some cases.
To protect personal privacy, each user perturbs her own data independently, and then sends the disguised data to an aggregator. Generally, the obfuscation mechanism corresponds to a noisy channel because -LDP is defined by a probabilistic function. In this way, a fundamental correlation has been established between LDP and information theory. To better illustrate this correlation, we first give the following example.
Let P be an arbitrary probability distribution on discrete set X , and P be a finite set representing all possible probability distributions, denoted as P ∈ P. We assume that each personal data to be drawn independently from a potential distribution P ∈ P that it is not known by the aggregator but only knows the true distribution lies in P. Further, we consider strategic users and aggregator that know the strategic space of each other. In this case, an honest-but-curious aggregator aims to maximize the success probability of privacy inference. For convenience, some notations used in this paper are listed in Table 1.

C. MINIMAX PRIVACY PROBLEM
The privacy budget of LDP is a de facto standard for measuring privacy level. However, we noticed that the notion of -LDP provides the worst-case privacy guarantee, that is, it has the strongest hypothesis of background knowledge for privacy attacker. Thus, this metrics has its drawbacks [14] in some cases, since is only determined by the probabilistic function mapping (defined by definition 1). If a set of privacy mechanisms all provide -privacy guarantee, then the -metrics can not distinguish which mechanism is better than others. In many applications, the qualities of privacy protection for these mechanisms need to be evaluated. The information-theoretic approach is an effective way to solve this problem. We present the method in details, which begins with a definition.
Definition 2 (Equivalent -privacy mechanisms): Let Q be a finite set representing a set of privacy-preserving mechanisms where contains k mechanisms. If each mechanism of Q is a -privacy mechanism, then these mechanisms are called equivalent -privacy mechanisms.
Remark 2: The condition of definition 2 can be relaxed to obtain a relaxing LDP mechanism set, that is, an arbitrary privacy mechanism Q i ∈ Q is i -LDP mechanism. In fact, the privacy mechanism Q i : X → Y is a lossy compression mechanism, which controls how many bits of private information flowing from real data to disguised data. To quantify the amount of information, we borrow an information-theoretic method, and define the aggregator's information gain as Definition 3: For a given private information x i , the probability distribution P(X = x i ) and P(X = x i |Y = y i ) represent the prior and posterior distribution after observing y i , respectively. The ratio log P(X =x i |Y =y i ) is defined as the aggregator's information gain.
From the definition 3 above, we can measure the amount of uncertainty reduction about real data after the aggregator has observed the disguised data. In fact, such metrics is a comparison between the priori and posterior probability of the real data. What's more, we noticed that this metrics has the same form with the well-known MI in information theory. Furthermore, the expected MI measures the information of a user loses on average, which can be used to measure information leakage of a privacy mechanism, i.e., MI leakage. Based on the notion of MI leakage, we argue that the equivalent -privacy mechanisms is comparable with each other. To demonstrate a partially ordered relation, we give the following definition. VOLUME 8, 2020 Definition 4: For a given prior probability distribution P, and arbitrary two privacy mechanisms More especially this relation is transitive, and it can be used to compare the privacy protection intensity of different mechanisms. Next, we consider the meaning of MI for privacy-preserving mechanism. First, the MI measures privacy leakage, which focus on the uncertainty of the real data given the disguised data. Second, the disguised data should preserve the information content of real data as much as possible while meeting the LDP constraints. Further, the information content in disguised data about real data is measured by the well-known MI [7]. Based on these theoretical supports, we consider the rational user aims to decrease the MI between real data and disguised data so that the aggregator can not have enough information to complete identify a user's personal data. However, the rational aggregator wants to maximize the privacy leakage to get more private information. From the analysis above, we can formulate the objective of users as the following minimax problem, such as inf Q∈Q sup P∈P I (P; Q).
In addition, the aggregator will estimate a distribution that maximizes MI because the set of priori distribution is available for him. In this case, the worst-case MI leakage of any privacy mechanism will be sup P∈P inf Q∈Q I (P; Q).
In fact, the above problem is formulated as a minimax problem, which becomes an convex optimization problem [44].
The minimax problem captures a basic scenario, where the players' goals are just opposite. In practice, the aggregator may be an strategic player rather than limited to observing the disguised data, who can change his own strategy according to the user's protection strategy. In such case, we consider MI leakage as the gain of the aggregator.

V. GAME MODEL AND ANALYSIS
In this section, we formulate the minimax privacy problem as a two-person zero-sum game, and further provide the theoretic analysis.

A. PRIVACY-PRESERVING ATTACK AND DEFENCE GAME MODEL
Each user perturbs her real data using privacy-preserving mechanism, who is analogous to a defender. As such, the aggregator is analogous to an attacker. Analogy-based, the above minimax problem naturally evolves into an attack and defence game problem. To have a better presentation, we first provide a formal definition. Definition 5: The privacy-preserving attack and defense (PPAD) game-theoretic framework is a tuple (D, A, D, A , U ), where D and A are the strategic space of the privacy defender D and attacker A respectively, and U : D × A → R is a von Neumann-Morgenstern utility function. Then, the rational behaviors both for them can be defined as In the above definition 5, we present a standard description about the PPAD game. To explain in details, we provide the game description of our PPAD, including players, strategic space and payment. Firstly, the players of PPAD game are attacker and defender. Secondly, the defender aims to decrease the private information loss, i.e., the desired information gain of attacker. Thus, we define a set of privacy mechanisms as the strategic space of the defender, denoted as D Q. Besides, we define all of possible distributions P on X as attacker's strategic space, denoted as A P. Thirdly, we define MI as the payoff function. To be specific, for any P ∈ P and Q ∈ Q, the payment is calculated by In the above PPAD game, the private information loss of the defender is the gaining of the attacker, which means that the goal of the defender is to minimize the loss, while the attacker aims to maximize the payment. Thus, the proposed PPAD is a two-person zero-sum (TPZS) game. We make a remark regarding the proposed game model. The saddle point strategy of PPAD game also provides a certain level of differential privacy. This is because the available actions of the defender are -privacy mechanisms. Hence, PPAD guarantees certain level of differential privacy, and is determined by the saddle point strategy. To have a better illustration of our idea, we provide an example.
Example 2: Assume the set of source distribution P contains 3 different distributions on source alphabet with |X | = 3, denoted as P i ∈ P, i ∈ {1, 2, 3}. The instances are given in Table 2. Moreover, we consider that the set of privacy mechanisms Q also includes 3 different mechanisms, denoted as Table 3 shows them in details. As such, the above PPAD game is an instance of matrix games.
In this paper, we consider a simultaneous game with perfect information, which means that each player makes a decision without knowing the decision made by the other. In the PPAD game, we consider that the players' strategic actions and payoff function are common knowledge that they are known both by the attacker and the defender. In this case, we analyze the rational actions for the players of PPAD game. In fact, a solution of the strategic game is captured by the saddle point. In the following, we provide the theoretic analysis for the proposed PPAD game.

B. CONVEX-CONCAVE ANALYSIS
The well-known convex-concave game has a special form that the payoff function is a convex function of one player's actions, and it is a concave function of the other's actions [45]. In such games, the solutions are given by the pure strategies for each player.
For our PPAD game model, the strategies of attacker and defender are probabilistic sets, that is, they are convex sets. Therefore, for any convex combination of P 1 , P 2 ∈ P, the U (P) satisfies a concave function of a closed convex set. In addition, for any Q 1 , Q 2 ∈ Q, and a parameter λ ∈ R + , 0 < λ < 1, then their convex combination Q λ = λQ 1 + (1 − λ)Q 2 also satisfies -differential privacy, which has been proved in [13]. As a result, for each P, the payoff function is a convex function of Q. Based on these theoretic analysis, the game that we proposed in this paper is a convex-concave game.
Lemma 1: If U : P × Q → R is a concave function of P, then attacker has an optimal response strategy such that max P∈P min Q∈Q U (P, Q). Similarly, if it is a convex function of Q, then the defender has an optimal response strategy such that min Q∈Q max P∈P U (P, Q).
The proof of Lemma 1 is similar to the proof of Theorem 5.2 in [45], thus we omit this proof for space limitation.
In addition to the mentioned above, we noticed that the PPAD game is a simultaneous game with complete information, thus each player can predict other's optimal response strategy, i.e. dominant strategy. As a result, no matter an attacker or defender will have an optimal response strategy for the other's strategy. Based on this result, we have the following Theorem 2.
Proof: For arbitrary Q 1 , Q 2 ∈ Q, and a parameter λ ∈ R + (0 < λ < 1), their convex combination Q λ = λQ 1 + (1 − λ)Q 2 is also a -LDP [13]. Because that both P and Q are probability distribution sets, thus they are convex subset of Euclidean space. Furthermore, U (P, Q) is a function of two variables, which is concave in P for each Q and convex in Q for each P [42]. What's more, the finite sets of P and Q are compact, i.e. closed and bounded. Then, according to the well-known minimax theorem, the PPAD game exists a saddle-point.
From the Theorem 2 above, we can see that the saddle-point of proposed PPAD game is an extremal status of privacy leakage, which is the worst-case for privacy defender. In addition, Equation (6) indicates that the payment of saddle-point is the minimum information gain that an attacker can obtain from the real data. At the same time, this payment is the defender's maximum possible information loss. Therefore, the saddle point of PPAD can be used to assess mutual information (MI) leakage. Indeed, the corresponding payment is an upper bound of MI leakage.

C. GAME ANALYSIS
Game analysis aims to find the solutions of games, which is one of the major research objectives in the game theory. It is well-known that the solution of game is a steady state that each player has no incentive to deviate from this state, i.e., no player wants to change his current strategy. Based on the above Lemma 1, the strategy profile of saddle point needs to be the optimal response strategy that both for each player. In fact, the proposed PPAD game is a TPZS game with finite strategies. From the Theorem 2, our PPAD game has a saddle point because the convexity and concavity of the payoff function. In calculating the saddle point, the calculation is an iterative optimization problem between two convex sets. Inspired by this idea, we propose an algorithm to calculate the solution of the established PPAD game.
The procedures of solving maximin problem is an alternating optimization, and it is similar to the problem of minimizing distance between two convex sets. The basic idea is alternate to calculate an optimal response strategy between two convex sets, which mainly includes three steps:

Input:
Strategic actions P, Q and payoff function U (P, Q) Output: Saddle point (P * , Q * ) and its payment SD 1: Initialize set S 1 ← Q 0 with an arbitrary Q 0 ∈ Q 2: Calculate P * via Equation (8) 3: Calculate Q * by Equation (7) 4: while (P * , Q * ) is not a saddle point do 5: Calculate P * via Equation (8), and update P * to recalculate U (P * , Q * ) by Equation (10) 6: if (P * , Q * ) is saddle point then 7: return (P * , Q * ) and SD ← U (P * , Q * ) 8: else 9: Calculate Q * = arg min Q∈Q\S 1 U (P * , Q), and U (P * , Q * ) by Equation (10) 10: Update set S 1 ← S 1 Q * 11: end if 12: end while Step 1: For a single arbitrary strategy of defender, the attacker calculates an optimal response strategy satisfying arg max P∈P U (P, Q); VOLUME 8, 2020 Step 2: Further, the defender predicts the attacker's preference, thus the defender would like to take the action that satisfies arg min Q∈Q max P∈P U (P, Q); Step 3: Finally, alternating procedure updates their strategy choices, and repeats the above steps until a strategy profile (P * , Q * ) that is optimal both for the attacker and defender.
Next, we present these calculation procedures in a algorithm, and provide the description in details.
The Algorithm 1 receives the structure of PPAD game, including strategic spaces P and Q and payoff function U (P, Q). Then, it performs the calculations to output the saddle point and payment. Firstly, it initializes an arbitrary strategy Q 0 ∈ Q, and calculates an optimal response strategy for Q 0 by using Equation (8) (lines 1 ∼ 2 of Algorithm 1). Secondly, it calculates an optimal response strategy of defender by Equation (7), which is used to defend attacker's strategy (line 3 of Algorithm 1). Thirdly, it repeats these procedures of alternating optimization until a stable state (P * , Q * ) that are both optimal for the attacker and defender (lines 4 ∼ 13 of Algorithm 1). Finally, it returns the saddle point and corresponding payment.
To understand our algorithm intuitively, we provide the explanation using Example 2 and illustrate the payments in Table 4. We demonstrate these procedures by assuming the algorithm begins with the strategy Q 1 , then, the attacker prefers to take P 3 for the purpose to obtain a maximum payment 0.0662, i.e., P 3 is an optimal strategy for the attacker. Further, the defender predicts the attacker's action, and takes Q 2 to minimize the privacy loss. That is to say, the defender desires to achieve 0.0315, thus Q 2 is the defender's optimal strategy to defend P 3 . Meanwhile, P 3 is also the optimal strategy for defender's strategy Q 2 . Therefore, the strategy profile (P 3 , Q 2 ) is a saddle point of the PPAD game, which has a payment 0.0315. Also, the saddle point strategy guarantees = ln 2 differential privacy. Additionally, we depict the procedures of rational decision in Figure 2 to have a better illustration. We provide the computation complexity of Algorithm 1 by analyzing some fundamental operations. First, the attacker's strategic space P is searched to find an optimal response strategy P in the first iteration. Second, the algorithm calculates the optimal reaction Q to the attacker's strategy, which searches the strategic space of the defender. Finally, the termination condition guarantees the solution of maxmin problem. It is clear then that the cost grows with the sizes of P and Q. As long as both P and Q are finite, the whole procedures will be efficient.

VI. EXPERIMENTAL SIMULATIONS AND ANALYSIS
In this section, we illustrate the experimental results of our scheme, and further provide the analytic results. We implement the algorithm in Java and conduct our experiments on a PC running Win 10 OS.

A. CASE STUDY
For the case of |X | = |Y | = 6, we assume that the prior probability distribution lies in a certain set but does not know the true distribution exactly. To have an illustration intuitively, we borrow several distributions from [13], and show them in Table 5. Furthermore, we consider two alternative privacy mechanisms with = ln 2. Their probability density functions are shown in Table 6, where Q 1 is the truncated 1 2 -geometric mechanism [46] and Q 2 is a privacy mechanism that proposed in [23]. Further, we consider the well-known k-RR mechanism [7] that its diagonal probabilities are e /(|X | − 1 + e ). The k-RR provides = ln 2 differential privacy guarantee, if and only if its probability density function Q 3 satisfies We noticed that the mechanisms of {Q 1 , Q 2 , Q 3 } are equivalent ln 2-privacy mechanisms. To compare these privacy mechanisms, we assume they are possible privacy strategies of the privacy defender. In this case, we provide the analytic result below.
Based on these available actions, we analyze the rational behaviors of the attacker and defender. The corresponding game is solved by Algorithm 1. As a result, the algorithm outputs a saddle point (P 1 , Q 3 ) and payment 0.0351, which  means the MI privacy leakage would not exceed an upper bound (0.0351). In other cases, the defender has an incentive to change his current strategy. For instance, when we consider the uniform prior distribution, the payment of game will be 0.0633. In summary, these results indicate that the optimal privacy preserving mechanism is related to the prior distribution.
What's more, we solve the problem that not being able to compare between equivalent privacy preserving mechanisms by using the information-theoretic approach. For instance, taking the uniform prior distribution into consideration, MI privacy leakage of these mechanisms are in fact strict orderings, i.e. Q 1 = 0.5074 > Q 2 = 0.2164 > Q 3 = 0.0633. In fact, these numbers of MI leakage describe the defender's preference for different outcomes. Thus, we have Q 3 Q 2 Q 1 . This strict ordering provides an effective evaluation method for equivalent -privacy mechanisms.

B. NUMERICAL SIMULATION
In order to obtain the numerical simulation results, we randomly generate 10 different distributions for |X | = 6 and |X | = 12, respectively. Then, we use the randomized response to implement the privacy-preserving mechanism. Following [7], we set ranging from 0 to 10 that we can obtain a set of privacy mechanisms. Based on these simulation data, we provide the following analysis.
We assume that the distributions (resp. mechanisms) are available actions of the attacker (resp. defender). Further, we conduct the experiment on these generated data, and perform Algorithm 1 to calculate the saddle point of PPAD game. To overcome the effect of randomness, we compare the average performance measured by the normalized mutual information I (X ; Y )/H (X ) for all privacy mechanisms, i.e. privacy payment. In our PPAD game, the rational privacy attacker (resp. defender) prefers to take the strategy that maximizes (resp. minimizes) the outcome of game. In fact, MI privacy leakage is monotonicity about , thus the curve of normalized mutual information can be drawn with increasing.
The experimental results are shown in Figure 3. We can see that the MI privacy leakage of saddle point is the worst-case privacy leakage for privacy defender. Besides, Figure 3(a) and Figure 3(b) are confirm that the conclusion is not sensitive to the size of |X | because two experimental results have the same tendency. This worst-case MI leakage can help to assess privacy disclosure risks and choose the adaptive under the tolerable breach of privacy.

VII. CONCLUSION AND FUTURE WORK
In this work, we have formalized the problem of trade-off between privacy and utility as a minimax problem, and proposed the PPAD game-theoretic framework using information-theoretic approach. In particular, the established PPAD game is a TPZS game. To find the solution, we proposed an alternating optimization algorithm to compute the saddle point. Then, we demonstrated our scheme that can be used to compare the performance between equivalent privacy mechanisms. Further, we illustrated our privacy measure is the worst-case privacy leakage, that is, maximum privacy leakage for privacy defender.
In the future work, there still exists several interesting questions that are worthy to be further investigated. For example, the defender first takes action, and then the attacker takes action after observing the defender's action, thus the game model will evolve into a Steinberg game or dynamic game model of incomplete information. Moreover, all participators are assumed to be rational players to investigate the privacy-preserving mechanism design, which is also a fascinating topic.
NINGBO WU received the M.S. degree from the School of Information, Guizhou University of Finance and Economics, China, in 2016. He is currently pursuing the Ph.D. degree with the College of Computer Science and Technology, Guizhou University, China. He is also a Scholar with the State Key Laboratory of Public Big Data. His research interests include privacy preserving, data security, coding and information theory, and game theory.
CHANGGEN PENG received the Ph.D. degree from the College of Computer Science and Technology, Guizhou University, China, in 2007. He is currently a Professor and a Ph.D. Supervisor with the College of Computer Science and Technology, Guizhou University. He is also an Academic Leader of data security and cryptography with the State Key Laboratory of Public Big Data. His research interests include data privacy, cryptography, and big data technology and security.
KUN NIU received the B.S. and M.S. degrees from the China University of Mining and Technology, in 2008 and 2011, respectively. She is currently pursuing the Ph.D. degree with the College of Computer Science and Technology, Guizhou University, China. Her research interests include big data security, privacy computing, and privacy protection. VOLUME 8, 2020