Logic, Self-awareness and Self-improvement: the Metacognitive Loop and the Problem of Brittleness

This essay describes a general approach to building perturbation-tolerant autonomous systems, based on the conviction that artificial agents should be able to notice when something is amiss, assess the anomaly, and guide a solution into place. This basic strategy of self-guided learning is termed the metacognitive loop; it involves system monitoring, reasoning about, and, when necessary, altering its own decision-making components. This paper (a) argues that equipping agents with a metacognitive loop can help to overcome the brittleness problem, (b) details the metacognitive loop and its relation to ongoing work on time-sensitive commonsense reasoning, (c) describes specific, implemented systems whose perturbation tolerance was improved by adding a metacognitive loop, and (d) outlines both short-term and long-term research agendas.