Towards Requirements Specification for Machine-learned Perception Based on Human Performance

The application of machine learning (ML) based perception algorithms in safety-critical systems such as autonomous vehicles have raised major safety concerns due to the apparent risks to human lives. Yet assuring the safety of such systems is a challenging task, in a large part because ML components (MLCs) rarely have clearly specified requirements. Instead, they learn their intended tasks from the training data. One of the most well-studied properties that ensure the safety of MLCs is the robustness against small changes in images. But the range of changes considered small has not been systematically defined. In this paper, we propose an approach for specifying and testing requirements for robustness based on human perception. With this approach, the MLCs are required to be robust to changes that fall within the range defined based on human perception performance studies. We demonstrate the approach on a state-of-the-art object detector.