Multipurpose Deep-Learning Accelerator for Arbitrary Quantization With Reduction of Storage, Logic, and Latency Waste | IEEE Journals & Magazine | IEEE Xplore

Multipurpose Deep-Learning Accelerator for Arbitrary Quantization With Reduction of Storage, Logic, and Latency Waste


Abstract:

Various pruning and quantization heuristics have been proposed to compress recent deep-learning models. However, the rapid development of new optimization techniques make...Show More

Abstract:

Various pruning and quantization heuristics have been proposed to compress recent deep-learning models. However, the rapid development of new optimization techniques makes it difficult for domain-specific accelerators to efficiently process various models showing irregularly stored parameters or nonlinear quantization. This article presents a scalable-precision deep-learning accelerator that supports multiply-and-accumulate operations (MACs) with two arbitrarily quantized data sequences. The proposed accelerator includes three main features. To minimize logic overhead when processing arbitrarily quantized 8-bit precision data, a lookup table (LUT)-based runtime reconfiguration is proposed. The use of bit-serial execution without unnecessary computations enables the multiplication of data with non-equal precision while minimizing logic and latency waste. Furthermore, two distinct data formats, raw and run-length compressed, are supported by a zero-eliminator (ZE) and runtime-density detector (RDD) that are compatible with both formats, delivering enhanced storage and performance. For a precision range of 1–8 bit and fixed sparsity of 30%, the accelerator implemented in 28 nm low-power (LP) CMOS shows a peak performance of 0.87–5.55 TOPS and a power efficiency of 15.1–95.9 TOPS/W. The accelerator supports processing with arbitrary quantization (AQ) while achieving state-of-the-art (SOTA) power efficiency.
Published in: IEEE Journal of Solid-State Circuits ( Volume: 59, Issue: 1, January 2024)
Page(s): 143 - 156
Date of Publication: 02 October 2023

ISSN Information:

Funding Agency:

Author image of Seunghyun Moon
Institute of Artificial Intelligence, Pohang University of Science and Technology, Pohang, South Korea
Seunghyun Moon (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Institute of Artificial Intelligence, POSTECH. His research interests include low-power and energy-efficient deep-learning ha...Show More
Seunghyun Moon (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Institute of Artificial Intelligence, POSTECH. His research interests include low-power and energy-efficient deep-learning ha...View more
Author image of Han-Gyeol Mun
department of Electrical Engineering, Pohang University of Science and Technology, Pohang, South Korea
Han-Gyeol Mun (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Department of Electronic and Electrical Engineering, POSTECH. His research primarily focuses on developing low-power and energ...Show More
Han-Gyeol Mun (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Department of Electronic and Electrical Engineering, POSTECH. His research primarily focuses on developing low-power and energ...View more
Author image of Hyunwoo Son
department of Electronic Engineering, Gyeongsang National University, Jinju, South Korea
Hyunwoo Son (Member, IEEE) received the B.S. and Ph.D. degrees in electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2012 and 2019, respectively.
From 2019 to 2020, he was with the Institute of Microelectronics, Agency for Science, Technology, and Research (A*STAR), Singapore, where he was involved in the design of biomedical integrated circuits. In 2020, he joine...Show More
Hyunwoo Son (Member, IEEE) received the B.S. and Ph.D. degrees in electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2012 and 2019, respectively.
From 2019 to 2020, he was with the Institute of Microelectronics, Agency for Science, Technology, and Research (A*STAR), Singapore, where he was involved in the design of biomedical integrated circuits. In 2020, he joine...View more
Author image of Jae-Yoon Sim
department of Electrical Engineering, Pohang University of Science and Technology, Pohang, South Korea
Jae-Yoon Sim (Senior Member, IEEE) received the B.S., M.S., and Ph.D. degrees in electronic and electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 1993, 1995, and 1999, respectively.
From 1999 to 2005, he was a Senior Engineer with Samsung Electronics, Hwasung, South Korea. From 2003 to 2005, he was a Post-Doctoral Researcher at the University of Southern Californ...Show More
Jae-Yoon Sim (Senior Member, IEEE) received the B.S., M.S., and Ph.D. degrees in electronic and electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 1993, 1995, and 1999, respectively.
From 1999 to 2005, he was a Senior Engineer with Samsung Electronics, Hwasung, South Korea. From 2003 to 2005, he was a Post-Doctoral Researcher at the University of Southern Californ...View more

Author image of Seunghyun Moon
Institute of Artificial Intelligence, Pohang University of Science and Technology, Pohang, South Korea
Seunghyun Moon (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Institute of Artificial Intelligence, POSTECH. His research interests include low-power and energy-efficient deep-learning hardware design for edge IoT devices.
Seunghyun Moon (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Institute of Artificial Intelligence, POSTECH. His research interests include low-power and energy-efficient deep-learning hardware design for edge IoT devices.View more
Author image of Han-Gyeol Mun
department of Electrical Engineering, Pohang University of Science and Technology, Pohang, South Korea
Han-Gyeol Mun (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Department of Electronic and Electrical Engineering, POSTECH. His research primarily focuses on developing low-power and energy-efficient deep-learning hardware solutions specifically tailored for edge IoT devices.
Han-Gyeol Mun (Member, IEEE) received the B.S. degree in electrical engineering and the Ph.D. degree in convergence IT engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2015 and 2023, respectively.
He is currently a Post-Doctoral Researcher with the Department of Electronic and Electrical Engineering, POSTECH. His research primarily focuses on developing low-power and energy-efficient deep-learning hardware solutions specifically tailored for edge IoT devices.View more
Author image of Hyunwoo Son
department of Electronic Engineering, Gyeongsang National University, Jinju, South Korea
Hyunwoo Son (Member, IEEE) received the B.S. and Ph.D. degrees in electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2012 and 2019, respectively.
From 2019 to 2020, he was with the Institute of Microelectronics, Agency for Science, Technology, and Research (A*STAR), Singapore, where he was involved in the design of biomedical integrated circuits. In 2020, he joined Gyeongsang National University, Jinju, South Korea, where he is currently an Assistant Professor. His research interests include sensor interface circuits, biomedical circuits, and neuromorphic circuits.
Hyunwoo Son (Member, IEEE) received the B.S. and Ph.D. degrees in electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 2012 and 2019, respectively.
From 2019 to 2020, he was with the Institute of Microelectronics, Agency for Science, Technology, and Research (A*STAR), Singapore, where he was involved in the design of biomedical integrated circuits. In 2020, he joined Gyeongsang National University, Jinju, South Korea, where he is currently an Assistant Professor. His research interests include sensor interface circuits, biomedical circuits, and neuromorphic circuits.View more
Author image of Jae-Yoon Sim
department of Electrical Engineering, Pohang University of Science and Technology, Pohang, South Korea
Jae-Yoon Sim (Senior Member, IEEE) received the B.S., M.S., and Ph.D. degrees in electronic and electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 1993, 1995, and 1999, respectively.
From 1999 to 2005, he was a Senior Engineer with Samsung Electronics, Hwasung, South Korea. From 2003 to 2005, he was a Post-Doctoral Researcher at the University of Southern California, Los Angeles, CA, USA. From 2011 to 2012, he was a Visiting Scholar with the University of Michigan, Ann Arbor, MI, USA. In 2005, he joined POSTECH, where he is currently a Professor. Since 2019, he has been the Director of the Scalable Quantum Computer Technology Platform Center, Pohang, South Korea, nominated by the Korean Ministry of Science and Information and Communication Technology (ICT). His research interests include sensor interface circuits, high-speed serial/parallel links, phase-locked loops, data converters, brain-inspired computing, and quantum computing.
Dr. Sim was a co-recipient of the Takuo Sugano Award from the IEEE International Solid-State Circuits Conference (ISSCC) in 2001 and a recipient of the Special Author Recognition Award from ISSCC 2013. In 2020, he received the Scientist of the Month Award sponsored by the Ministry of Science and ICT of Korea. He has been an IEEE Distinguished Lecturer from 2018 to 2019. He has served on the Technical Program Committees for the ISSCC, the IEEE Symposium on VLSI Circuits, and the IEEE Asian Solid-State Circuits Conference.
Jae-Yoon Sim (Senior Member, IEEE) received the B.S., M.S., and Ph.D. degrees in electronic and electrical engineering from the Pohang University of Science and Technology (POSTECH), Pohang, South Korea, in 1993, 1995, and 1999, respectively.
From 1999 to 2005, he was a Senior Engineer with Samsung Electronics, Hwasung, South Korea. From 2003 to 2005, he was a Post-Doctoral Researcher at the University of Southern California, Los Angeles, CA, USA. From 2011 to 2012, he was a Visiting Scholar with the University of Michigan, Ann Arbor, MI, USA. In 2005, he joined POSTECH, where he is currently a Professor. Since 2019, he has been the Director of the Scalable Quantum Computer Technology Platform Center, Pohang, South Korea, nominated by the Korean Ministry of Science and Information and Communication Technology (ICT). His research interests include sensor interface circuits, high-speed serial/parallel links, phase-locked loops, data converters, brain-inspired computing, and quantum computing.
Dr. Sim was a co-recipient of the Takuo Sugano Award from the IEEE International Solid-State Circuits Conference (ISSCC) in 2001 and a recipient of the Special Author Recognition Award from ISSCC 2013. In 2020, he received the Scientist of the Month Award sponsored by the Ministry of Science and ICT of Korea. He has been an IEEE Distinguished Lecturer from 2018 to 2019. He has served on the Technical Program Committees for the ISSCC, the IEEE Symposium on VLSI Circuits, and the IEEE Asian Solid-State Circuits Conference.View more

Contact IEEE to Subscribe

References

References is not available for this document.