SambaNova SN10 RDU: A 7nm Dataflow Architecture to Accelerate Software 2.0

The availability of large amounts of data and advances in modern machine-learning algorithms are pushing the limits of computing systems and redefining the way software is written. This new software paradigm, termed “Software 2.0”, is a departure from deterministic computing - centered around exact specifications and expert-created algorithms to accomplish a task - to probabilistic computing, centered around methods that learn to accomplish the same task using several examples. Software 2.0 contains a graph of operations that is rich in data locality and has abundant data, task, and hierarchical pipeline parallelism. Consequently, Software 2.0 can be accelerated by building custom dataflow pipelines. However, conventional GPU systems provide limited flexibility to build such dataflow pipelines. As a result, they suffer from poor device utilization and require a high-bandwidth off-chip memory system, which results in lower memory capacity. Memory capacity limitations impose serious challenges for increasingly larger models and data sets common in the fields of Natural Language Processing (NLP), high-resolution computer vision, and large recommender systems. SambaNova Systems Cardinal SN10 is a Reconfigurable Dataflow Unit (RDU) that enables accelerating Software 2.0 with the flexibility to build custom dataflow pipelines as well as large memory capacity to run big models efficiently [1].