Bridging Divides: Unifying AI Architectures from from Edge to Cloud

The world of artificial intelligence and machine learning (AI/ML) is fragmented into different domains, split by training and inference, and cloud versus edge. Training normally takes place in the cloud where many high-powered servers, plenty of memory, hardware accelerators, and high-speed networking can be thrown at the workload. Scaling-out workloads typically results in communication bottlenecks which can be alleviated with novel compute architectures where part of the AI/ML workload and network functions such as collectives are handled in the network. Inference workloads can also be performed in a data center, but increasingly, inference tasks are migrating to the client devices at the edge because of latency, bandwidth and privacy reasons. Most edge inference devices cannot afford the silicon or the power consumption to perform all calculations using full-precision, floating-point data formats and require more efficient utilization of the compute HW.AI/ML accelerator scalable architectures that support both full- and reduced-precision numerical formats break down the artificial boundary between training and inference, cloud, network and edge, and enable the deployment of the same standard and familiar software tools for a unified architecture. These efficient edge AI accelerators employ architectural innovations such as dataflow and on-chip broadcast networks that permit data fetched from external memory to be reused many times once brought on-chip, thus saving power and scaling performance. The availability of a unified scalable dataflow architecture for machine learning breaks down the wall between distinctive phases of training and inference and edge vs cloud. The unified architecture ensures that compatible data formats are used and optimizations for data-formats such as sparsity representations don’t break between cloud and edge. A scalable, unified architecture and continual learning throughout the lifetime of a deployed application departs from today’s conventional training and inference practice that relies on CPUs and GPUs in the data center and specialized devices in the edge. Yet, this unified approach seems the most logical path if the industry wants to make large gains in performance, accuracy, and power efficiency as AI/ML becomes pervasive from edge to cloud.