Exploring Noise- Resilient Spiking Neural Encoding using ∑†∑ Neurons

Researchers are increasingly focusing on spiking neurons to enhance the energy efficiency of edge machine learning (ML) models. Spiking neural encoding has evolved from traditional methods like Integrate and Fire and Time to First Spike to techniques such as Delta and Sigma-delta modulation (SD), enabling sparser and energy-efficient feature representation. In this work, we introduce the Sigma-Delta-Sigma (∑†∑/SDS) neuron, a noise-invariant spiking neural encoding technique. Our aim with this neural encoding is to emulate the robustness of a specific class of biological neurons to input noise by effectively filtering out noise-like features from the input stimuli. While noise injection in training prevents overfitting, improper noise profiles can be detrimental to the inference accuracy of a model. Our objective with the SDS encoding is to develop noise-resilient spiking neural network models capable of being trained with ideal features, while still being able to extract features from low-quality data during the inference phase. To assess the robustness of our technique, we implement an information encoding ensemble model that demonstrates a 6.2x improvement in robustness when the Signal-to-Noise Ratio (SNR) of the incoming signal is 1 dB. Furthermore, we evaluate the performance of the SDS neurons in more complex models such as a Liquid State Machine (LSM), where the model exhibits a 3.87% improvement in the predictive accuracy against the baseline model when tested against input features with SNR degradation from 55 to 1 dB.