Integral Quadratic Constraints for Neural Networks

The formalism of Integral Quadratic Constraints (IQCs) is well-established in robust control. It has recently been used for systems with a neural network as one of its components, however, using only a small subset of the established techniques for obtaining IQC relations. We provide a larger set of IQCs relevant for the nonlinearities commonly used in neural networks, introduce new constraints for the rectified linear unit and the leaky rectified linear unit, and draw on the established literature to build a library of IQCs to use in connection with neural networks. Finally, our examples show how improved guarantees can be obtained with a larger library of I QCs.