Collaborative Control of Aerial Robots for Inferring Human Intent from Gaze Following

In an ideal human-robot collaboration, autonomous robots work side-by-side with humans in a joint workspace, often performing complementary tasks to the humans. A robotic ability to infer human intention and goals directly from human behavior will facilitate the collaboration and maximize its efficiency. In this paper, we focus on inferring which object the human wants picked up next, based on what the human is looking at, by visually following the human gaze and head orientation. We develop a coordination protocol for a team of aerial robots to extract effective human head and gaze cues. The aerial robots are controlled to navigate around the human and collect data that improves the detection of the human's gaze and hence the intended object to be picked up. The effectiveness of the approach is shown using simulations in AirSim, a photo-realistic simulator.