MOSA: Matrix Optimized Self-Attention Hardware Accelerator for Mobile Device

The Self-Attention mechanism, which lies at the core of Transformer architectures, plays a vital role in capturing long-range dependencies. However, its high computational complexity and significant memory requirements pose major challenges for resource-constrained hardware such as mobile devices. In particular, frequent memory accesses and inefficient matrix multiplication operations often result in performance bottlenecks. Therefore, the development of dedicated hardware accelerators for Self-Attention, focusing on optimizing matrix operations and reducing memory usage, is essential for improving AI processing efficiency on mobile devices. This paper presents a Self-Attention hardware accelerator designed specifically for mobile devices. By optimizing the matrix multiplication process, the accelerator effectively reduces data transmission and memory access frequency, thereby lowering the overall computational complexity. It also reuses intermediate computation results, minimizing frequent memory read and write operations, which significantly reduces memory bandwidth requirements. This data reuse strategy not only cuts down on redundant computations but also greatly enhances computational efficiency. In addition, the accelerator eliminates the need for Key Transpose operations, further simplifying certain computational steps. Compared to traditional algorithms, the idle time is reduced by 66.5%., and signal line transmission costs are reduced by approximately 80%. With an optimized hardware architecture, mobile devices can efficiently support complex deep learning applications, such as speech processing and image recognition.