Roads Digital Twin: Predictive Situational Awareness Using 360$^{\circ }$ Video Streaming and Graph Neural Networks

As vehicle technologies rapidly advance, video streaming capabilities emerge as a significant feature of modern on-board vehicular systems. The integration of 360$^{\circ }$ video streaming in vehicles is expected to enhance road situational awareness, by providing panoramic live streaming and recording capabilities. This paper introduces the concept of roads digital twin, combining, for the first time, 360$^{\circ }$ video streaming with graph neural networks, in order to enhance predictive situational awareness in road environments. To this end, we have developed eGAT, a selective edge-enhanced graph attention network architecture, that uses graph neural networks with attention mechanisms. eGAT is capable of effectively predicting road coverage, considering uplink bandwidth limitations that may affect video streaming quality and user quality of experience (QoE). For the evaluation of our novel method, we utilized four different datasets, considering several vehicular scenarios. For performance comparison purposes, we employed three metrics, namely the overall percentage of the covered region, the normalized mutual information (NMI), and the precision-recall scores. In terms of overall coverage percentage, eGAT provided superior coverage performance against similar studies in all nine scenarios and for all numbers of streaming vehicles investigated, reaching an increase of up to 32.3%. In terms of NMI score, for low values of prediction horizon, eGAT outperformed similar attention-based algorithms, with an increase of up to 29.6%, whereas for larger prediction horizons, eGAT presented lower, however comparable, coverage performance. Our results suggest that eGAT is a promising solution for scenarios involving streaming vehicles, showcasing its potential for applications, such as autonomous vehicles and traffic management systems.