Multi-Agent Reinforcement Learning for Freshness-Aware Data Sensing Model in Vehicular Crowdsensing Systems

Vehicular Crowdsensing (VCS) is a promising paradigm for supporting urban sensing services, where Service Providers (SPs) engage Mobile Vehicles (MVs) to perform data sensing tasks with specific objectives. However, existing studies have predominantly focused on data sensing quality in terms of data collection completeness and geographic fairness, while largely neglecting the important aspect of data freshness. Moreover, effective mechanisms for optimizing data freshness through coordination of the behaviors of both SPs and MVs are still lacking. Accordingly, this paper proposes a Freshness-Aware Data Sensing (FDS) model by considering heterogeneous data freshness, varying sensing capabilities of MVs, and limited budgets of SPs. The FDS is formulated as a two-stage game model, where SPs and MVs iteratively determine their pricing and sensing strategies in a self-interested manner to maximize their individual gains. Further, we develop a multi-agent reinforcement learning-based approach to learn the pricing strategies based on historical observations, which allows SPs to make pricing decisions without global knowledge. Additionally, given the pricing strategies of SPs, the optimal solution for each MV is derived. Finally, we build the simulation model based on realistic vehicular traces, where the simulation results demonstrate the superiority of the proposed algorithm in various scenarios.