On the representation of piecewise quadratic functions by neural networks

Neural networks (NNs) are commonly used to approximate functions based on data samples, as they are a universal function approximator for a large class of functions. However, choosing a suitable topology in terms of depth, width and activation function for NNs that allow for low error approximations is a non-trivial task. For the approximation of continuous piecewise affine (PWA) functions, this task has been solved by showing that for every PWA function, there exist NNs with rectified linear unit (relu) and maxout activation that allow an exact representation of the PWA function. This connection between PWA functions and NNs has led to some valuable insights into the representation capabilities of NNs. Moreover, the connection was used in control for approximating the PWA optimal control law of model predictive control (MPC) for linear systems. We show that a similar connection exists between NNs and continuous piecewise quadratic (PWQ) functions by deriving topologies for NNs that allow an exact representation of arbitrary PWQ functions with a polyhedral domain partition. Furthermore, we demonstrate that the proposed NNs can efficiently approximate the PWQ optimal value function for linear MPC.