Efficient Matrix Engine Design Based on Collaborative Small-Scale Systolic Arrays

The growing scale of machine learning models has sharply increased the demand for computational resources. Traditional general-purpose processors (CPUs) struggle to support the intensive workloads required by modern deep learning applications. To improve matrix computation performance, recent CPU designs from Intel and IBM have integrated dedicated matrix engines. Among these, largescale systolic arrays are commonly used for their high data reuse and low control overhead. However, their fixed structure limits flexibility and results in inefficient handling of irregular or small matrix operations, thereby reducing overall resource utilization.To overcome these limitations, we propose PoMX, a multi-mode systolic array matrix engine. PoMX coordinates multiple small-scale systolic arrays operating in parallel to reduce data transfer latency and improve resource efficiency. In addition, the engine applies sub-array grouping and on-demand activation to better allocate computational resources and control power consumption. Experimental results demonstrate that PoMX can deliver up to 2.68x performance improvement over traditional large-scale systolic array designs when tested on deep learning workloads, under identical resource conditions.