Designing Compatible Analog Circuits for Equilibrium Propagation: Implementations Using The Adjoint Method and Reciprocity Principles

Equilibrium Propagation (EP) offers an energyefficient alternative to backpropagation for training analog neural networks, enabling on-device learning in Edge AI applications. However, practical hardware implementations of EP are hindered by stringent circuit requirements and challenges in computing loss gradients for common functions like crossentropy without resorting to expensive data converters. In this paper, we address these challenges by identifying the necessary conditions that an analog circuit must satisfy to implement EP, leveraging the adjoint method and the reciprocity principle of analog circuits, and developing two novel circuit architectures that fully realize EP in hardware. The first architecture employs the sparsemax activation function, offering a practical means to implement a crossentropy-like loss function while circumventing the complexities associated with the softmax function. The second architecture introduces a simpler topology that aligns with analog processing principles by enforcing output values to reside in a simplex, eliminating the need for calculating probabilities explicitly. We validate our designs on two benchmark datasets, demonstrating that our circuits can effectively and efficiently train analog neural networks using EP within practical hardware constraints.