An oversampled A/D converter with on-chip reinforcement learning

The quality and stability of noise shaping is a concern in the design of higher-order delta-sigma modulators for high-resolution, high-speed oversampled analog-to-digital conversion. We reformulate noise-shaping modulation alternatively as a nonlinear optimal control problem, where the objective is to find the binary modulation sequence that minimizes signal swing in a cascade of integrators operating on the difference between the input signal and the modulation sequence. We use reinforcement learning to adaptively optimize an on-line nonlinear neural classifier which outputs modulation bits from the values of the input signal and integration state variables. Analogous to the classical pole balancing control problem, a punishment signal triggers learning whenever any of the integrators saturate. We train a simple classifier consisting of locally tuned, binary address-encoded neurons to produce noise shaping modulation of order one and two, and present experimental learning results obtained from an analog VLSI modulator containing an array of 64 neurons for on-chip classification and learning.