Adaptive Camera Platform Using Deep Learning-Based Early Warning of Task Failures

There is a growing interest in deploying complex deep neural networks (DNN) in smart sensors to extract task-specific information from real-time sensor data. This paper presents an adaptive sensor that uses a novel light-weight deep learning platform, WarningNet, to estimate potential task failures due to perturbations in the sensor data and control its read-out-circuit (ROIC) parameters such as sampling resolution and operating voltage. Simulation results show that WarningNet can provide early warning of the performance degradation of various tasks within a fraction of the time required for the task to complete. Moreover, the early warning guided adaptive control of sampling resolution and operating voltage of the ROIC of a digital pixel sensor shows potential of dynamic trade off between task accuracy and sensor energy/bandwidth.