Socially-Compliant Hierarchical Human–Vehicle Collaboration With Multimodal Haptic Steering

Advancements in autonomous driving technologies continue to revolutionize transportation, yet the full realization of self-driving vehicles remains hampered by several critical challenges. Automated vehicles continue to encounter significant challenges in perception, prediction, and decision-making, while their low-level modules are relatively mature and robust. Conversely, humans surpass machines in terms of high-level intelligence but may suffer from control performance degradation. To coalesce the strength of the human and machine, a novel collaboration scheme is proposed to compensate for the prediction-decision uncertainty via human guidance while providing low-level control feedback to the driver. This approach generates multiple decision candidates and corresponding predictions for other road users using a transformer-based socially compliant generative adversarial network (SCGAN). The driver can assist in choosing the appropriate candidate using the context-understanding capability, while concurrently, the control projection of this chosen decision guides the driver to achieve the desired objective via haptic steering feedback. The haptic feedback can reflect the decision uncertainties enabled by the decision-control projection of the intention estimation of the ego vehicle. A Type-II fuzzy controller is utilized to determine the control authority to account for the complexity of the future movement. We verify the effectiveness of the proposed algorithm through a real-time human-in-the-loop experiment, including an ablation study and comparisons with other human-machine collaboration schemes. The results demonstrate that the proposed scheme can minimize human-machine conflicts while increasing system safety.