Reinforcement Learning for Intelligent Sensor Virtualization and Provisioning in Internet of Vehicles (IoV)

The Internet of Vehicles (IoV) is a powerful application of the Internet of Things (IoT) in the Intelligent Transportation System (ITS). It enables device connectivity, and interaction with the environment, and improves efficiency in utilizing sensor data. Leveraging the capabilities of vehicle sensors to provide virtual sensor service presents an opportunity to make the most of underutilized sensor resources and offer on-demand sensor services. However, challenges remain in ensuring spatio-temporal sensor availability due to the high mobility of vehicles and frequent changes in the IoV network topology. Furthermore, providing sensor services in the dynamic IoV-Cloud market, while balancing cost-effectiveness for consumers and profitability for service providers, poses a significant challenge. To address these challenges, we develop an IoV-Cloud architecture tailored for vehicle virtual sensor provisioning, integrating interactive functional layers and components for a comprehensive solution. This architecture supports vehicle sensor virtualization and management, facilitating the intelligent provisioning of on-demand, elastic, and scalable vehicle virtual sensors from existing physical sensors within mobile vehicles. To address vehicle mobility and enhance sensor availability, our design permits virtual sensors to maintain data provision even when switched to different physical sensors. Further optimizing sensor utilization, our system allows multiple virtual vehicle sensors to share a single physical sensor by employing a configuration mapping mechanism. At the heart of our strategy is a reinforcement learning model that dynamically selects the most suitable vehicle physical sensor for each time slot throughout the service period, considering both physical availability and cost-effectiveness for the Sensor Cloud Service Provider (SCSP). We simulate the proposed intelligent sensor selection model using both Q-learning and SARSA algorithms, demonstrating its effectiveness in intelligent and dynamic sensor provisioning.