Conceptualisation of Human-on-the-Loop Haptic Teleoperation With Fully Autonomous Self-Driving Vehicles in the Urban Environment

The automotive industry aims to deploy commercial level-5 fully autonomous self-driving vehicles (FA-SDVs) in a diverse range of benefit-driven concepts on city roads in the years to come. In all future visions of operating networks of FA-SDVs, humans are expected to intervene with some kind of remote supervisory role. Recent advances in cyber-physical systems (CPS) within the concept of Internet of Everything (IoE) using tactile Internet (TI) teleport us to teleoperate remote objects within the cyber-world. Human-on-the-loop (HOTL) haptic teleoperation with an extension of human control and sensing capability by coupling with artificial sensors and actuators with an increased sense of real-time driving in the remote vehicle can help overcome the challenging tasks when the new driver — artificial intelligence (AI) agent — encounters an unorthodox situation that can’t be addressed by the autonomous capabilities. This paper analyses HOTL real-time haptic delay-sensitive teleoperation with FA-SDVs, in the aspects of human-vehicle teamwork by establishing two similar remote parallel worlds — real-world vehicle time-varying environment and cyber-world emulation of this environment, i.e., digital twins (DTs) — in which a human telesupervisor (HTS), as a biological agent, can be immersed with no cybersickness enabling omnipresence through a timely bidirectional flow of energy and information. The experiments conducted as a proof of concept of HOTL haptic teleoperation regarding learning with human-vehicle collaboration show promising results and the potential of benefiting from the proposed framework.