ROMA: A Reconfigurable On-chip Memory Architecture for Multi-core Accelerators

Designing efficient on-chip memory solutions for multi-core accelerators is essential to mitigating the impact of the memory wall and enhancing their overall performance. Among prevalent on-chip memory solutions, scratchpad memory (SPM) and cache stand out. Unfortunately, both SPM and cache excel only in specific scenarios, lacking broad applicability across various scenarios. In this paper, we propose a Reconfigurable On-chip Memory Architecture (ROMA) that combines the benefits of SPM and cache. First, we decouple the prefetchable data access from the application execution and describe an ISA that can support explicit decoupled prefetch and postback. Then, we introduce an on-chip memory microarchitecture for multi-core accelerators, reducing the area overhead by utilizing the reconfigurable physical storage shared by SPM and cache. Finally, we present a mechanism for data mapping and on-chip memory partitioning in an application targeted to ROMA-based architectures, facilitating efficient data classification into prefetchable and non-prefetchable categories. Experimental results show that our method achieves an average access latency decrease of 34.8 % and up to 17.1 % access energy reduction compared with existing on-chip memory architectures across a range of applications.