The human should be part of the control loop?

The capabilities of autonomy have grown to encompass new application spaces that until recently were considered exclusive to humans. In the past, automation has focused on applications where it was preferable to completely replace the human. Today, though, we have the opportunity to leverage the complementary strengths of both human and autonomy technologies to maximize performance and limit risk, and the human should therefore remain “in” or “on” the loop. To adequately assess when and how to accomplish this, it requires us to assess not only the capabilities, but the risks and the ethical questions; coupled to this are the issues with degradation of performance in specific instances (for instance, recovery from failure) that may require a human to remain the sole control authority. This paper investigates the contributors to success/failure in current human-autonomy integration frameworks, and proposes guidelines for safe and resilient use of humans and autonomy with regard to performance, consequence, and the stability of human-machine switching. Key to our proposed approach are (i) the relative error rate between the human and autonomy and (ii) the consequence of possible events.