Strengthened Circle and Popov Criteria and the Analysis of ReLU Neural Networks

Many systems involving neural networks (NNs) can be framed as Lurie systems: feedback systems consisting of a linear time-invariant (LTI) part and a static nonlinearity. Examples of these include the interconnection of LTI systems with L-layer feedforward NNs [1], [2] and continuous time recurrent neural networks (RNN) [3]. Stability analysis of a Lurie system lends itself to a range of criteria from absolute stability; however, in NN analysis, the size of $m$ (see Fig. 1 where $u, y\in\Re^{m}$) is typically large. As a result, existing absolute stability criteria suffer from greater conservatism and/or computational complexity [4]. This paper addresses this problem by strengthening the low complexity classical Circle and Popov Criteria for the specialised case of the repeated ReLU nonlinearity (a popular NN activation function). The results are cast as a set of linear matrix inequalities (LMIs) with less restrictive conditions on the matrix variables than their classical counterparts. A full version of this paper has recently been in published in [5].