Conduct Governance Assurance - An Architecture Implementation for High-Risk Autonomous Systems

As we place robotic systems into more complex environments and introduce increased reliance on autonomous decision making, trust in autonomous robotic planners becomes essential. Real-world environments can require a system to moderate its actions according to legal or ethical requirements as well as the usual functional requirements. In this paper, we present an implementation of a reference architecture for a robotic system that integrates conduct governance within the perception, tracking, and response pipeline. We define conduct governance as the concept that an autonomous system can interpret a set of rules for its operating domain, and implement components that are designed to act as a safeguard to ensure that autonomous decisions are aligned with these rules and ethical and safety-oriented system requirements. We implemented the reference’s patterns and practices to react to visual targets, while adhering to changing sets of high-level conduct rules. We found that, compared to traditional uses of machine learning models, the conduct governance architecture enables a higher level of observability and traceability into autonomous and intelligent systems, and provides tunable guardrails around systems that may be running one or more machine learning models. We also present lessons learned from implementing such an architecture.