A Three-Party Hierarchical Game for Physical Layer Security Aware Wireless Communications With Dynamic Trilateral Coalitions

In this paper, a novel hierarchical game framework for physical layer security (PLS) aware wireless communications with dynamic trilateral coalitions is studied. In the considered system, legitimate users (LUs) aim to transmit secret data to associated base stations (BSs) via uplink communications under the threat of eavesdroppers (EVs), while there also exist jammers (JAs) which may choose to form coalitions with either LUs for increasing their secrecy transmission rates or EVs for increasing their eavesdropping rates in exchange for potential rewards. Different from the existing work, we explore such complicated while dynamic coalition relationships under uncertainties of wireless systems (e.g., time-varying channel conditions), and formulate a hierarchical game integrated with a dynamic trilateral coalition formation game to model strategic interactions among LUs, JAs and EVs. Particularly, we first analyze stability requirements of the trilateral coalitions and propose a hedonic coalition selection and formation algorithm for reaching a stable coalition partition in each time slot. In addition, we propose a deep reinforcement learning (DRL) based solution, which can achieve the equilibrium with long-term performance guarantees for the hierarchical game running over multiple time slots with dynamic evolutions. Simulations evaluate the proposed solution and show its superiority over its counterparts.