Opportunities and Challenges for Flagman Recognition in Autonomous Vehicles

Autonomous vehicles promise significant advances in transportation safety, efficiency and comfort. However, achieving the goal of full autonomy is impeded by the need to address several operational challenges encountered in practice. Gesture recognition of flagmen on roads is one such set of challenges. An autonomous vehicle needs to make safe decisions and facilitate forward progress in the presence of road construction workers and flagmen. However, human gestures under diverse environmental conditions are very varied and represent significant complexity. In this work, we present (i) a taxonomy of challenges for organizing traffic gestures, (ii) a sizeable flagman gesture dataset, and (iii) extensive experiments on practical algorithms for gesture recognition. We categorize traffic gestures according to their semantics, flagman appearances and the environmental context. We then collect a dataset covering a range of common flagman gestures with and without props such as signs and flags. Finally, we develop a recognition algorithm using different feature representations of the human pose and perform extensive ablation experiments on each component.