Mixed Autonomous Supervision in Traffic Signal Control

Traffic signal control is a critical component for ensuring smooth traffic flows in city corridors. To this end, deep reinforcement learning (RL) agents have recently been proposed. These learned black-box policies may out-perform manually tuned policies on average, but they remain imperfect and come at a cost of how easy they are to interpret and supervise. These challenges hinder their adoption in real-world systems. To address the first challenge, this paper devises naturally interpretable decision tree policies that imitate expert deep neural network policies. To address supervision, we define a new formalization called Mixed Autonomous Supervision (MAS), which concerns integrating an imperfect policy into an existing supervision system. We propose a two-part supervision model with online automated supervision and offline human supervision to implement MAS. We present a novel blind spot detection algorithm for decision tree policies to encourage the safe transfer of control to an automated fail-safe policy (online supervision) and an interactive dashboard DTLight for offline human supervision. We show the decision tree policies are just as performant as the RL policies, and the proposed supervision model has a significant benefit in scenarios derived from real traffic situations.