Challenges of Testing Cognitive EW Systems

This paper describes some of the test infrastructure requirements for testing the performance of artificial intelligence (AI) systems that can learn from novel experiences in the field. As a test community, we must validate the learning process, rather than validating the learned model. Cognitive capabilities will force multiple paradigm shifts from legacy test and evaluation (T&E) approaches. Notably, the system may never be the same twice. Testing must therefore become a continuous process, rather than a final exam, where we develop tests to understand system limitations and range of effectiveness. These paradigm shifts highlight some of the improvements needed in the current test infrastructure. The paper describes some of the requirements that should be placed on cognitive systems for data collection, transparency, and rules for how to interact with legacy systems. We use Cognitive Electronic Warfare (CogEW) as the motivating scenario, where AI enables EW systems to respond more quickly and effectively to real-world conditions with complex and novel emitters.