Optimizing Intelligent Edge-clouds with Partitioning, Compression and Speculative Inference

Internet of Battlefield Things (IoBTs) are well positioned to take advantage of recent technology trends that have led to the development of low-power neural accelerators and low-cost high-performance sensors. However, a key challenge that needs to be dealt with is that despite all the advancements, edge devices remain resource-constrained, thus prohibiting complex deep neural networks from deploying and deriving actionable insights from various sensors. Furthermore, deploying sophisticated sensors in a distributed manner to improve decision-making also poses an extra challenge of coordinating and exchanging data between the nodes and server. We propose an architecture that abstracts away these thorny deployment considerations from an end-user (such as a commander or warfighter). Our architecture can automatically compile and deploy the inference model into a set of distributed nodes and server while taking into consideration of the resource availability, variation, and uncertainties.