Abstract:
This work considers the problem of computing the canonical polyadic decomposition (CPD) of large tensors. Prior works leverage data sparsity to handle this problem, which...Show MoreMetadata
Abstract:
This work considers the problem of computing the canonical polyadic decomposition (CPD) of large tensors. Prior works leverage data sparsity to handle this problem, which is not suitable for handling dense tensors that often arise in applications such as medical imaging, computer vision, and remote sensing. Stochastic optimization is known for its low memory cost and per-iteration complexity when handling dense data. However, existing stochastic CPD algorithms are not flexible to incorporate a variety of constraints/regularization terms that are of interest in signal and data analytics. Convergence properties of many such algorithms are also unclear. In this work, we propose a stochastic optimization framework for large-scale CPD with constraints/regularization terms. The framework works under a doubly randomized fashion, and can be regarded as a judicious combination of randomized block coordinate descent (BCD) and stochastic proximal gradient (SPG). The algorithm enjoys lightweight updates and small memory footprint. This framework entails considerable flexibility-many frequently used regularizers and constraints can be readily handled. The approach is supported by convergence analysis. Numerical results on large-scale dense tensors are presented to showcase the effectiveness of the proposed approach.
Published in: IEEE Transactions on Signal Processing ( Volume: 68)
Funding Agency:
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, USA
Xiao Fu (Member, IEEE) received the Ph.D. degree in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2014. He is an Assistant Professor with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, USA. He was a Postdoctoral Associate with the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, USA, from...Show More
Xiao Fu (Member, IEEE) received the Ph.D. degree in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2014. He is an Assistant Professor with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, USA. He was a Postdoctoral Associate with the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, USA, from...View more
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, USA
Shahana Ibrahim received the B.Tech. degree in electronics and communication engineering from National Institute of Technology, Calicut, India, in 2012, and the M.S. degree in electrical engineering in 2019 from Oregon State University, Corvallis, OR, USA, where she is currently working toward the Ph.D. degree. She had been working as a System Validation Engineer with Texas Instruments, Bengaluru, India, from 2012 to 2017...Show More
Shahana Ibrahim received the B.Tech. degree in electronics and communication engineering from National Institute of Technology, Calicut, India, in 2012, and the M.S. degree in electrical engineering in 2019 from Oregon State University, Corvallis, OR, USA, where she is currently working toward the Ph.D. degree. She had been working as a System Validation Engineer with Texas Instruments, Bengaluru, India, from 2012 to 2017...View more
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, Hong Kong
Hoi-To Wai (Member, IEEE) received the Ph.D. degree in electrical engineering from Arizona State University (ASU), Tempe, AZ, USA, in Fall 2017, and the B. Eng. (with First Class Hons.) and M. Phil. degrees in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2010 and 2012, respectively. He is an Assistant Professor with the Department of Systems Engineering and Engineering Management, ...Show More
Hoi-To Wai (Member, IEEE) received the Ph.D. degree in electrical engineering from Arizona State University (ASU), Tempe, AZ, USA, in Fall 2017, and the B. Eng. (with First Class Hons.) and M. Phil. degrees in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2010 and 2012, respectively. He is an Assistant Professor with the Department of Systems Engineering and Engineering Management, ...View more
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, USA
University of Missouri-Columbia, Columbia, USA
Cheng Gao received the bachelor‘s degree in automation from Wuhan University of Technology, Wuhan, China, in 2016 and the master's degree in electrical and computer engineering from Oregon State University, Corvallis, OR, USA, in 2019, with research focus on large-scale tensor decomposition algorithms. He is currently working toward the Ph.D. degree in computer science with the University of Missouri, Columbia, MO, USA. H...Show More
Cheng Gao received the bachelor‘s degree in automation from Wuhan University of Technology, Wuhan, China, in 2016 and the master's degree in electrical and computer engineering from Oregon State University, Corvallis, OR, USA, in 2019, with research focus on large-scale tensor decomposition algorithms. He is currently working toward the Ph.D. degree in computer science with the University of Missouri, Columbia, MO, USA. H...View more
Department of Computer and Information Science and Engineering, University of Florida, Gainesville, USA
Kejun Huang (Member, IEEE) received the B.Eng. degree in communication engineering from Nanjing University of Information Science and Technology, Nanjing, China, in 2010 and the Ph.D. degree in electrical engineering from the University of Minnesota, Minneapolis, MN, USA, in 2016. He is currently an Assistant Professor with the Department of Computer and Information Science and Engineering, University of Florida, Gainesvi...Show More
Kejun Huang (Member, IEEE) received the B.Eng. degree in communication engineering from Nanjing University of Information Science and Technology, Nanjing, China, in 2010 and the Ph.D. degree in electrical engineering from the University of Minnesota, Minneapolis, MN, USA, in 2016. He is currently an Assistant Professor with the Department of Computer and Information Science and Engineering, University of Florida, Gainesvi...View more
Contents School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, USA
Xiao Fu (Member, IEEE) received the Ph.D. degree in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2014. He is an Assistant Professor with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, USA. He was a Postdoctoral Associate with the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, USA, from 2014 to 2017. His research interests include the broad area of signal processing and machine learning. He received a Best Student Paper Award at ICASSP 2014. Two of his coauthored papers received Best Student Paper Awards at IEEE CAMSAP 2015 and IEEE MLSP 2019, respectively. He serves as the Treasurer of IEEE Signal Processing Society Oregon Chapter. He also serves as a member of the EURASIP Technical Area Committee in Signal Processing for Multisensor Systems for the term of 2020–2023.
Xiao Fu (Member, IEEE) received the Ph.D. degree in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2014. He is an Assistant Professor with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, USA. He was a Postdoctoral Associate with the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN, USA, from 2014 to 2017. His research interests include the broad area of signal processing and machine learning. He received a Best Student Paper Award at ICASSP 2014. Two of his coauthored papers received Best Student Paper Awards at IEEE CAMSAP 2015 and IEEE MLSP 2019, respectively. He serves as the Treasurer of IEEE Signal Processing Society Oregon Chapter. He also serves as a member of the EURASIP Technical Area Committee in Signal Processing for Multisensor Systems for the term of 2020–2023.View more
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, USA
Shahana Ibrahim received the B.Tech. degree in electronics and communication engineering from National Institute of Technology, Calicut, India, in 2012, and the M.S. degree in electrical engineering in 2019 from Oregon State University, Corvallis, OR, USA, where she is currently working toward the Ph.D. degree. She had been working as a System Validation Engineer with Texas Instruments, Bengaluru, India, from 2012 to 2017. Her research interests are in the broad areas of statistical machine learning and signal processing.
Shahana Ibrahim received the B.Tech. degree in electronics and communication engineering from National Institute of Technology, Calicut, India, in 2012, and the M.S. degree in electrical engineering in 2019 from Oregon State University, Corvallis, OR, USA, where she is currently working toward the Ph.D. degree. She had been working as a System Validation Engineer with Texas Instruments, Bengaluru, India, from 2012 to 2017. Her research interests are in the broad areas of statistical machine learning and signal processing.View more
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, Hong Kong
Hoi-To Wai (Member, IEEE) received the Ph.D. degree in electrical engineering from Arizona State University (ASU), Tempe, AZ, USA, in Fall 2017, and the B. Eng. (with First Class Hons.) and M. Phil. degrees in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2010 and 2012, respectively. He is an Assistant Professor with the Department of Systems Engineering and Engineering Management, CUHK. He has held research positions with ASU, UC Davis, Telecom ParisTech, Ecole Polytechnique, LIDS, MIT. His research interests are in the broad area of signal processing, machine learning and distributed optimization, with a focus of their applications to network science. His dissertation has received the 2017's Dean's Dissertation Award from the Ira A. Fulton Schools of Engineering of ASU and he was the recipient of Best Student Paper Award at ICASSP 2018.
Hoi-To Wai (Member, IEEE) received the Ph.D. degree in electrical engineering from Arizona State University (ASU), Tempe, AZ, USA, in Fall 2017, and the B. Eng. (with First Class Hons.) and M. Phil. degrees in electronic engineering from The Chinese University of Hong Kong (CUHK), Hong Kong, in 2010 and 2012, respectively. He is an Assistant Professor with the Department of Systems Engineering and Engineering Management, CUHK. He has held research positions with ASU, UC Davis, Telecom ParisTech, Ecole Polytechnique, LIDS, MIT. His research interests are in the broad area of signal processing, machine learning and distributed optimization, with a focus of their applications to network science. His dissertation has received the 2017's Dean's Dissertation Award from the Ira A. Fulton Schools of Engineering of ASU and he was the recipient of Best Student Paper Award at ICASSP 2018.View more
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, USA
University of Missouri-Columbia, Columbia, USA
Cheng Gao received the bachelor‘s degree in automation from Wuhan University of Technology, Wuhan, China, in 2016 and the master's degree in electrical and computer engineering from Oregon State University, Corvallis, OR, USA, in 2019, with research focus on large-scale tensor decomposition algorithms. He is currently working toward the Ph.D. degree in computer science with the University of Missouri, Columbia, MO, USA. He is currently focusing his research on sparse learning and bioinformatics.
Cheng Gao received the bachelor‘s degree in automation from Wuhan University of Technology, Wuhan, China, in 2016 and the master's degree in electrical and computer engineering from Oregon State University, Corvallis, OR, USA, in 2019, with research focus on large-scale tensor decomposition algorithms. He is currently working toward the Ph.D. degree in computer science with the University of Missouri, Columbia, MO, USA. He is currently focusing his research on sparse learning and bioinformatics.View more
Department of Computer and Information Science and Engineering, University of Florida, Gainesville, USA
Kejun Huang (Member, IEEE) received the B.Eng. degree in communication engineering from Nanjing University of Information Science and Technology, Nanjing, China, in 2010 and the Ph.D. degree in electrical engineering from the University of Minnesota, Minneapolis, MN, USA, in 2016. He is currently an Assistant Professor with the Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL, USA. He was a Postdoctoral Associate with the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, from 2016 to 2018. His research interests include machine learning, signal processing, optimization, and statistics.
Kejun Huang (Member, IEEE) received the B.Eng. degree in communication engineering from Nanjing University of Information Science and Technology, Nanjing, China, in 2010 and the Ph.D. degree in electrical engineering from the University of Minnesota, Minneapolis, MN, USA, in 2016. He is currently an Assistant Professor with the Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL, USA. He was a Postdoctoral Associate with the Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, from 2016 to 2018. His research interests include machine learning, signal processing, optimization, and statistics.View more
