Reinforcement Contract Design for Vehicular-Edge Computing Scheduling and Energy Trading via Deep Q-Network With Hybrid Action Space

The advancements in information and communication technology have led to the emergence of innovative edge computing models that incorporate the computing power of vehicles into the energy sector. Electric vehicles (EVs), functioning as edge computing nodes, offer flexible computing offloading services for charging stations (CS). However, coordinating EV computing and charging should consider the interdependence with CS's specific computing requirements due to information asymmetry. Additionally, it is crucial to consider EV's charging demands and their social distance to computing tasks. In this context, it is natural to view EVs and CSs as self-interested prosumers who prioritize their individual utilities. To address the integration of strategic EV-CS interactions and uncertainties into the joint computing scheduling and energy trading, this paper proposes a parameterized deep Q-network-based reinforcement contract design framework, which employs a hybrid action space to design contracts that facilitate CSs in pairing computing tasks and charging resources with EVs. The objective is to incentivize EV participation and maximize long-term social welfare by incorporating incentive compatibility, individual rationality constraints, and capacity constraints into the contract design. Experimental results demonstrate that the proposed framework surpasses parameterized deep deterministic policy gradient-based and greedy-based contract designs, and achieves near-optimal solutions by solving deterministic optimizations.