Backprop as Functor: A compositional perspective on supervised learning

A supervised learning algorithm searches over a set of functions
$A\rightarrow B$
parametrised by a space
$P$
to find the best approximation to some ideal function
$f:A\rightarrow B$
. It does this by taking examples
$(a, f(a))\in A\times B$
, and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent-with respect to a fixed step size and an error function satisfying a certain property-defines a monoidal functor from a category of parametrised functions to this category of update rules. A key contribution is the notion of request function. This provides a structural perspective on backpropagation, giving a broad generalisation of neural networks and linking it with structures from bidirectional programming and open games.