Investigation into Massively Parallel MIMD Architecture based IPU System through Application Benchmarking

We present a performance benchmarking analysis of Intelligent Processing Unit (IPU). IPU is a new architecture developed by Graphcore, specifically designed for handling Neural Network (NN) and Graph Computing (GC) workloads. IPU-POD16 is a 5U server solution, from Graphcore, consisting of four IPU servers and one host server. Each IPU server (IPU M2000) consists of 4 MK2 GC200 IPU. Our performance evaluation is focused on two aspects. First -To measure throughput, considering IPU-POD16 as a single machine, Second-To assess the scalability, by varying the number of IPUs assigned to current workload. For compute workload we used the Tensorflow implementation of ResNet-50 and performed training and validation with the Imagenet dataset. Experiments are conducted to measure the performance of the IPU system with respect to scaling, precision, network latency by devising three separate strategies. We reported throughput numbers (images/second) for training and validation, with both real data and synthetic data. Our initial conclusion is that applications are easy to port and run in the IPU platform, due to availability and support of popular frameworks like tensorflow and pytorch. In order to fully utilize the capabilities of the IPU hardware platform, advanced engineering techniques are required. In this case, we measured the performance by benchmarking with limited application varieties. More study and experimentation are required to fully understand and realize the power of the architecture (computation, networking). It would also be interesting to see how applications from other domains (other than AI/ML/DL) take benefit of this new platform.