Abstract:
This article presents a deep learning-based classifier IC for keyword spotting (KWS) in 65-nm CMOS designed using an algorithm-hardware co-design approach. First, a recur...Show MoreMetadata
Abstract:
This article presents a deep learning-based classifier IC for keyword spotting (KWS) in 65-nm CMOS designed using an algorithm-hardware co-design approach. First, a recurrent attention model (RAM) algorithm for the KWS task (the KeyRAM algorithm) is proposed. The KeyRAM algorithm enables accuracy versus energy scalability via a confidence-based computation (CC) scheme, leading to a 2.5× reduction in computational complexity compared to state-of-the-art (SOTA) neural networks, and is well-suited for in-memory computing (IMC) since the bulk (89%) of its computations are 4-b matrix-vector multiplies. The KeyRAM IC comprises a multi-bit multi-bank IMC architecture with a digital co-processor. A sparsity-aware summation scheme is proposed to alleviate the challenge faced by IMCs when summing sparse activations. The digital co-processor employs diagonal major weight storage to compute without any stalls. This combination of the IMC and digital processors enables a balanced tradeoff between energy efficiency and high accuracy computation. The resultant KWS IC achieves SOTA decision latency of 39.9 μs with a decision energy <; 0.5 μJ/dec which translates to more than 24 × savings in the energy-delay product (EDP) of decisions over existing KWS ICs.
Published in: IEEE Journal of Solid-State Circuits ( Volume: 56, Issue: 7, July 2021)
Funding Agency:
Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Hassan Dbouk (Student Member, IEEE) received the B.E. degree (Hons.) from the Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon, in 2017, and the M.S. degree from the Department of Electrical and Computer Engineering, University of Illinois at Urbana–Champaign, Urbana, IL, USA, where he is currently pursuing the Ph.D. degree.
His research interests lie at the intersection of ...Show More
Hassan Dbouk (Student Member, IEEE) received the B.E. degree (Hons.) from the Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon, in 2017, and the M.S. degree from the Department of Electrical and Computer Engineering, University of Illinois at Urbana–Champaign, Urbana, IL, USA, where he is currently pursuing the Ph.D. degree.
His research interests lie at the intersection of ...View more
Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Amazon, Seattle, WA, USA
Sujan K. Gonugondla (Member, IEEE) received the bachelor’s and master’s in technology degrees in electrical engineering from IIT Madras, Chennai, India, in 2014, and the Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2020.
Since June 2020, he has been with Amazon, Seattle, WA, USA, where he works as a Research Scientist. His research interests ar...Show More
Sujan K. Gonugondla (Member, IEEE) received the bachelor’s and master’s in technology degrees in electrical engineering from IIT Madras, Chennai, India, in 2014, and the Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2020.
Since June 2020, he has been with Amazon, Seattle, WA, USA, where he works as a Research Scientist. His research interests ar...View more
Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Charbel Sakr (Graduate Student Member, IEEE) received the Engineering degree (Hons.) from the American University of Beirut, Lebanon, in 2015, and the master’s degree from the Electrical and Computer Engineering Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2017, where he is currently pursuing the Ph.D. degree working with Prof. Naresh Shanbhag in the Coordinate Sciences Laboratory.
His resear...Show More
Charbel Sakr (Graduate Student Member, IEEE) received the Engineering degree (Hons.) from the American University of Beirut, Lebanon, in 2015, and the master’s degree from the Electrical and Computer Engineering Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2017, where he is currently pursuing the Ph.D. degree working with Prof. Naresh Shanbhag in the Coordinate Sciences Laboratory.
His resear...View more
Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Naresh R. Shanbhag (Fellow, IEEE) received the B.Tech. degree from IIT Delhi, New Delhi, India, in 1988, the M.S. degree from Wright State University, Dayton, OH, USA, in 1990, and the Ph.D. degree from the University of Minnesota, Minneapolis, MN, USA, in 1993, all in electrical engineering.
From 1993 to 1995, he worked at AT&T Bell Laboratories, Murray Hill, NJ, USA, where he was the Lead Chip Architect for AT&T’s 51.84-...Show More
Naresh R. Shanbhag (Fellow, IEEE) received the B.Tech. degree from IIT Delhi, New Delhi, India, in 1988, the M.S. degree from Wright State University, Dayton, OH, USA, in 1990, and the Ph.D. degree from the University of Minnesota, Minneapolis, MN, USA, in 1993, all in electrical engineering.
From 1993 to 1995, he worked at AT&T Bell Laboratories, Murray Hill, NJ, USA, where he was the Lead Chip Architect for AT&T’s 51.84-...View more
Contents Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Hassan Dbouk (Student Member, IEEE) received the B.E. degree (Hons.) from the Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon, in 2017, and the M.S. degree from the Department of Electrical and Computer Engineering, University of Illinois at Urbana–Champaign, Urbana, IL, USA, where he is currently pursuing the Ph.D. degree.
His research interests lie at the intersection of machine learning, circuits, and computer architecture.
Mr. Dbouk was a recipient of the Rambus Fellowship from the ECE Department, University of Illinois, in 2020–2021.
Hassan Dbouk (Student Member, IEEE) received the B.E. degree (Hons.) from the Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon, in 2017, and the M.S. degree from the Department of Electrical and Computer Engineering, University of Illinois at Urbana–Champaign, Urbana, IL, USA, where he is currently pursuing the Ph.D. degree.
His research interests lie at the intersection of machine learning, circuits, and computer architecture.
Mr. Dbouk was a recipient of the Rambus Fellowship from the ECE Department, University of Illinois, in 2020–2021.View more
Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Amazon, Seattle, WA, USA
Sujan K. Gonugondla (Member, IEEE) received the bachelor’s and master’s in technology degrees in electrical engineering from IIT Madras, Chennai, India, in 2014, and the Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2020.
Since June 2020, he has been with Amazon, Seattle, WA, USA, where he works as a Research Scientist. His research interests are in energy-efficient integrated circuits and low-complexity algorithms for machine learning systems, specifically algorithm hardware co-design for inference under resource-constraints.
Dr. Gonugondla was a recipient of the Dr. Ok Kyun Kim Fellowship 2018–2019, the M. E. Van Valkenburg Graduate Research Award 2019–2020 from the ECE Department, University of Illinois at Urbana–Champaign, the ADI Outstanding Student Designer Award 2018, and the SSCS Predoctoral Achievement Award in 2020. He has received Best Student Paper Awards in the International Conference on Acoustics, Speech and Signal Processing (ICASSP) in 2016 and the International conference in Circuits and Systems (ISCAS) in 2018.
Sujan K. Gonugondla (Member, IEEE) received the bachelor’s and master’s in technology degrees in electrical engineering from IIT Madras, Chennai, India, in 2014, and the Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2020.
Since June 2020, he has been with Amazon, Seattle, WA, USA, where he works as a Research Scientist. His research interests are in energy-efficient integrated circuits and low-complexity algorithms for machine learning systems, specifically algorithm hardware co-design for inference under resource-constraints.
Dr. Gonugondla was a recipient of the Dr. Ok Kyun Kim Fellowship 2018–2019, the M. E. Van Valkenburg Graduate Research Award 2019–2020 from the ECE Department, University of Illinois at Urbana–Champaign, the ADI Outstanding Student Designer Award 2018, and the SSCS Predoctoral Achievement Award in 2020. He has received Best Student Paper Awards in the International Conference on Acoustics, Speech and Signal Processing (ICASSP) in 2016 and the International conference in Circuits and Systems (ISCAS) in 2018.View more
Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Charbel Sakr (Graduate Student Member, IEEE) received the Engineering degree (Hons.) from the American University of Beirut, Lebanon, in 2015, and the master’s degree from the Electrical and Computer Engineering Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2017, where he is currently pursuing the Ph.D. degree working with Prof. Naresh Shanbhag in the Coordinate Sciences Laboratory.
His research interests are in resource-constrained machine learning, with a focus on analysis and implementation of reduced precision algorithms and models.
Charbel Sakr (Graduate Student Member, IEEE) received the Engineering degree (Hons.) from the American University of Beirut, Lebanon, in 2015, and the master’s degree from the Electrical and Computer Engineering Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA, in 2017, where he is currently pursuing the Ph.D. degree working with Prof. Naresh Shanbhag in the Coordinate Sciences Laboratory.
His research interests are in resource-constrained machine learning, with a focus on analysis and implementation of reduced precision algorithms and models.View more
Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Naresh R. Shanbhag (Fellow, IEEE) received the B.Tech. degree from IIT Delhi, New Delhi, India, in 1988, the M.S. degree from Wright State University, Dayton, OH, USA, in 1990, and the Ph.D. degree from the University of Minnesota, Minneapolis, MN, USA, in 1993, all in electrical engineering.
From 1993 to 1995, he worked at AT&T Bell Laboratories, Murray Hill, NJ, USA, where he was the Lead Chip Architect for AT&T’s 51.84-Mb/s transceiver chips over twisted-pair wiring for asynchronous transfer mode (ATM)-LAN and very high-speed digital subscriber line (VDSL) chip-sets. Since August 1995, he has been with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA, where he is presently a Jack Kilby Professor of electrical and computer engineering. He was a Visiting Faculty at National Taiwan University, Taipei, Taiwan, from August 2007 to December 2007 and Stanford University, Stanford, CA, USA, from August 2014 to December 2014. He holds 12 U.S. patents. He is also a coauthor of the research monograph Pipelined Adaptive Digital Filters published by Kluwer Academic Publishers in 1994. His research interests are in the design of robust and energy-efficient integrated circuits and systems for communications, including VLSI architectures for error-control coding and equalization, noise-tolerant integrated circuit design, error-resilient architectures and systems, and system-assisted mixed-signal design. He has more than 200 publications in these areas.
Dr. Shanbhag received the 2010 Richard Newton GSRC Industrial Impact Award, the 2006 IEEE Journal of Solid-State Circuits Best Paper Award, the 2001 IEEE Transactions on Very Large Scale Integration (VLSI) Systems Best Paper Award, the 1999 IEEE Leon K. Kirchmayer Best Paper Award, the 1999 Xerox Faculty Award, the Distinguished Lecturership from the IEEE Circuits and Systems Society in 1997, the National Science Foundation CAREER Award in 1996, and the 1994 Darlington Best Paper Award from the IEEE Circuits and Systems Society. He served as an Associate Editor for the IEEE Journal on Exploratory Solid-State Computation Devices and Circuits from 2014 to 2016, the IEEE Transaction on Circuits and Systems—II: Express Briefs from 1997 to 1999, and the Transactions on Very Large Scale Integration (VLSI) Systems from 1999 to 2002 and 2009 to 2011, respectively. He has served as the General Chair for the IEEE Workshop on Signal Processing Systems in 2013 and the IEEE International Symposium on Low-Power Design (ISLPED 2012), and the Technical Program Co-Chair for the 2010 ISLPED, and served on the Technical Program Committee of a number of conferences, including the International Solid-State Circuits Conference (ISSCC) from 2007 to 2011. He led the Alternative Computational Models in the Post-Si Era research theme, DOD, and Semiconductor Research Corporation (SRC) sponsored the Microelectronics Advanced Research Corporation (MARCO) center under their Focus Center Research Program (FCRP) from 2006 to 2012. Since January 2013, he has been the Founding Director of the Systems On Nanoscale Information fabriCs (SONIC) Center, a five-year multi-university center funded by DARPA and SRC under the STARnet phase of FCRP. In 2000, he co-founded and served as the Chief Technology Officer of Intersymbol Communications, Inc., a venture-funded fabless semiconductor startup that provides DSP-enhanced mixed-signal ICs for electronic dispersion compensation of OC-192 optical links. In 2007, Intersymbol Communications, Inc., was acquired by Finisar Corporation, Inc.
Naresh R. Shanbhag (Fellow, IEEE) received the B.Tech. degree from IIT Delhi, New Delhi, India, in 1988, the M.S. degree from Wright State University, Dayton, OH, USA, in 1990, and the Ph.D. degree from the University of Minnesota, Minneapolis, MN, USA, in 1993, all in electrical engineering.
From 1993 to 1995, he worked at AT&T Bell Laboratories, Murray Hill, NJ, USA, where he was the Lead Chip Architect for AT&T’s 51.84-Mb/s transceiver chips over twisted-pair wiring for asynchronous transfer mode (ATM)-LAN and very high-speed digital subscriber line (VDSL) chip-sets. Since August 1995, he has been with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University of Illinois at Urbana–Champaign, Urbana, IL, USA, where he is presently a Jack Kilby Professor of electrical and computer engineering. He was a Visiting Faculty at National Taiwan University, Taipei, Taiwan, from August 2007 to December 2007 and Stanford University, Stanford, CA, USA, from August 2014 to December 2014. He holds 12 U.S. patents. He is also a coauthor of the research monograph Pipelined Adaptive Digital Filters published by Kluwer Academic Publishers in 1994. His research interests are in the design of robust and energy-efficient integrated circuits and systems for communications, including VLSI architectures for error-control coding and equalization, noise-tolerant integrated circuit design, error-resilient architectures and systems, and system-assisted mixed-signal design. He has more than 200 publications in these areas.
Dr. Shanbhag received the 2010 Richard Newton GSRC Industrial Impact Award, the 2006 IEEE Journal of Solid-State Circuits Best Paper Award, the 2001 IEEE Transactions on Very Large Scale Integration (VLSI) Systems Best Paper Award, the 1999 IEEE Leon K. Kirchmayer Best Paper Award, the 1999 Xerox Faculty Award, the Distinguished Lecturership from the IEEE Circuits and Systems Society in 1997, the National Science Foundation CAREER Award in 1996, and the 1994 Darlington Best Paper Award from the IEEE Circuits and Systems Society. He served as an Associate Editor for the IEEE Journal on Exploratory Solid-State Computation Devices and Circuits from 2014 to 2016, the IEEE Transaction on Circuits and Systems—II: Express Briefs from 1997 to 1999, and the Transactions on Very Large Scale Integration (VLSI) Systems from 1999 to 2002 and 2009 to 2011, respectively. He has served as the General Chair for the IEEE Workshop on Signal Processing Systems in 2013 and the IEEE International Symposium on Low-Power Design (ISLPED 2012), and the Technical Program Co-Chair for the 2010 ISLPED, and served on the Technical Program Committee of a number of conferences, including the International Solid-State Circuits Conference (ISSCC) from 2007 to 2011. He led the Alternative Computational Models in the Post-Si Era research theme, DOD, and Semiconductor Research Corporation (SRC) sponsored the Microelectronics Advanced Research Corporation (MARCO) center under their Focus Center Research Program (FCRP) from 2006 to 2012. Since January 2013, he has been the Founding Director of the Systems On Nanoscale Information fabriCs (SONIC) Center, a five-year multi-university center funded by DARPA and SRC under the STARnet phase of FCRP. In 2000, he co-founded and served as the Chief Technology Officer of Intersymbol Communications, Inc., a venture-funded fabless semiconductor startup that provides DSP-enhanced mixed-signal ICs for electronic dispersion compensation of OC-192 optical links. In 2007, Intersymbol Communications, Inc., was acquired by Finisar Corporation, Inc.View more
