SSMA: A Memory-Efficient Accelerator for State Space Model in the Mamba

Mamba has exhibited great potential across various tasks, achieving the powerful capability of long-sequence modeling with linear complexity. Selective State Space Models (SSMs), the core component of Mamba, possess unique computational flow and massive memory requirements, which pose new challenges for deployment on edge devices and are difficult to be efficiently supported by existing deep learning accelerators. Therefore, we develop the first memory-efficient Selective SSM Accelerator, namely SSMA. Specifically, we design a reconfigurable hardware architecture and low-complexity nonlinear units to efficiently execute the rearranged and fused operations in Selective SSM. Moreover, we introduce a novel tile-stationary recurrent dataflow to achieve on-chip layer fusion by recurrently updating the tiled latent state and SSM parameters, dramatically decreasing memory usage and access. Experimental results show that the proposed SSMA, evaluated on Xilinx ZCU102 FPGA, achieves up to 3.33× speedup and 41.8 × higher energy efficiency than the optimized GPU implementation with the same setting.