Managing Timing Uncertainties in Worst-Case Design of Machine Learning Applications

Achieving reliable worst-case timing poses a challenge for modern, high-performance, commercial off-the-shelf hardware platforms deployed for industrial applications. Particularly for safety-critical industrial systems, e.g., robot-human collaboration using convolutional neural networks, timing must be considered to operate safely. Although state-of-the-art real-time operating systems and isolation techniques provide predictable timing, they restrict design decisions as many modern hardware platforms are not supported, introducing serious performance penalties. Besides traditional timing considerations, such as the number of cache misses, process variations due to chip manufacturing become more prominent, causing chips from the same model series to exhibit different timing behavior. This circumstance complicates achieving reliable timing on a system level even further. In this work, we present examples of physical variations using most recent hardware platforms, including 12th-generation Intel-based embedded hardware and GPU-based platforms using an Nvidia Jetson AGX Xavier. We elaborate on a potential solution from the avionics domain, called Timing Diversity, which allows for masking unexpected occurrences of worst-case timing behavior by exploiting modular redundancy inherent to safety-critical systems. A key result of our work is that Timing Diversity enables the safe usage of high-performance platforms such as the Nvidia Jetson AGX Xavier, consequently yielding a significant performance boost of nearly 6x.