Abstract:
Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., m...Show MoreMetadata
Abstract:
Eye tracking has become an essential human–machine interaction modality in virtual reality (VR) and augmented reality (AR) applications requiring high throughput (e.g., more than 240 frames per second), small form factor, and enhanced visual privacy. Existing eye tracking systems have adopted bulky, lens-based cameras, and thus suffer from both a large form factor and high communication cost between the camera and back-end processor. This work presents a camera, algorithm, and accelerator co-designed lensless eye tracking system dubbed EyeCoD, which, to the best of our knowledge, is the first to provide a general, front-end eye tracking solution for AR/VR while satisfying the requirements for both high throughput and smaller form factor. Specifically, EyeCoD integrates system-, algorithm-, and accelerator-level techniques to boost system efficiency without sacrificing eye tracking accuracy. We believe that our EyeCoD system will pave the way for next-generation eye tracking solutions in VR/AR and shed light on future innovations for intelligent imaging systems.
Published in: IEEE Micro ( Volume: 43, Issue: 4, July-Aug. 2023)
Funding Agency:
Georgia Institute of Technology, Atlanta, GA, USA
Haoran You is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. You received his master’s degree in electrical and computer engineering from Rice University. Contact him at haoran.you@gatech.edu.
Haoran You is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. You received his master’s degree in electrical and computer engineering from Rice University. Contact him at haoran.you@gatech.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Yang Zhao is a postdoctoral researcher at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research interests include computer architecture and chip design. Zhao received her Ph.D. degree in electrical and computer engineering from Rice University. Contact her at yangkatie.zhao@gmail.com.
Yang Zhao is a postdoctoral researcher at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research interests include computer architecture and chip design. Zhao received her Ph.D. degree in electrical and computer engineering from Rice University. Contact her at yangkatie.zhao@gmail.com.View more
Georgia Institute of Technology, Atlanta, GA, USA
Cheng Wan is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include graph neural network and system design. Wan received his master’s degree in electrical and computer engineering from Rice University. Contact him at chwan@gatech.edu.
Cheng Wan is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include graph neural network and system design. Wan received his master’s degree in electrical and computer engineering from Rice University. Contact him at chwan@gatech.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Zhongzhi Yu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research focuses on machine learning. Yu received his master’s degree in electrical and computer engineering from Columbia University, New York. Contact him at zyu401@gatech.edu.
Zhongzhi Yu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research focuses on machine learning. Yu received his master’s degree in electrical and computer engineering from Columbia University, New York. Contact him at zyu401@gatech.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Yonggan Fu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Fu received his master’s degree in electrical and computer engineering from Rice University. Contact him at yfu314@gatech.edu.
Yonggan Fu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Fu received his master’s degree in electrical and computer engineering from Rice University. Contact him at yfu314@gatech.edu.View more
Rice University, Houston, TX, USA
Jiayi Yuan is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Yuan received his bachelor’s degree in computer science from Tsinghua University. Contact him at jy101@rice.edu.
Jiayi Yuan is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Yuan received his bachelor’s degree in computer science from Tsinghua University. Contact him at jy101@rice.edu.View more
Rice University, Houston, TX, USA
Shang Wu is an M.S. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Wu received his master’s degree in computer science from Rice University. Contact him at sw99@rice.edu.
Shang Wu is an M.S. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Wu received his master’s degree in computer science from Rice University. Contact him at sw99@rice.edu.View more
Rice University, Houston, TX, USA
Shunyao Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Zhang received his master’s degree in computer science from Carnegie Mellon University. Contact him at sz74@rice.edu.
Shunyao Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Zhang received his master’s degree in computer science from Carnegie Mellon University. Contact him at sz74@rice.edu.View more
Rice University, Houston, TX, USA
Yongan Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on computer architecture. Zhang received his master’s degree in electrical and computer engineering from Rice University. Contact him at yz87@rice.edu.
Yongan Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on computer architecture. Zhang received his master’s degree in electrical and computer engineering from Rice University. Contact him at yz87@rice.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Chaojian Li is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Li received his bachelor’s degree in electrical and computer engineering from Tsinghua University. Contact him at cli851@gatech.edu.
Chaojian Li is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Li received his bachelor’s degree in electrical and computer engineering from Tsinghua University. Contact him at cli851@gatech.edu.View more
Rice University, Houston, TX, USA
Vivek Boominathan is a postdoctoral researcher at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Boominathan received his Ph.D. degree in electrical and computer engineering from Rice University. Contact him at vivekb@rice.edu.
Vivek Boominathan is a postdoctoral researcher at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Boominathan received his Ph.D. degree in electrical and computer engineering from Rice University. Contact him at vivekb@rice.edu.View more
Rice University, Houston, TX, USA
Ashok Veeraraghavan is a professor at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Veeraraghavan received his Ph.D. degree in computational imaging from the University of Maryland. He is a Fellow of IEEE. Contact him at vashok@rice.edu.
Ashok Veeraraghavan is a professor at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Veeraraghavan received his Ph.D. degree in computational imaging from the University of Maryland. He is a Fellow of IEEE. Contact him at vashok@rice.edu.View more
Meta Reality Labs, Redmond, WA, USA
Ziyun Li is a research scientist at Meta Reality Labs, USA. His research interests include chip design and AR/VR. Li received his Ph.D. degree in electrical and computer engineering from the University of Michigan. Contact him at liziyun@fb.com.
Ziyun Li is a research scientist at Meta Reality Labs, USA. His research interests include chip design and AR/VR. Li received his Ph.D. degree in electrical and computer engineering from the University of Michigan. Contact him at liziyun@fb.com.View more
Georgia Institute of Technology, Atlanta, GA, USA
Yingyan (Celine) Lin is an associate professor at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research focuses on efficient machine learning via cross-layer innovations from algorithm to hardware architecture down to chip design. Lin received her Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana-Champaign. She is a Member of IEEE. Contact her at celine.lin@...Show More
Yingyan (Celine) Lin is an associate professor at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research focuses on efficient machine learning via cross-layer innovations from algorithm to hardware architecture down to chip design. Lin received her Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana-Champaign. She is a Member of IEEE. Contact her at celine.lin@...View more
Contents Georgia Institute of Technology, Atlanta, GA, USA
Haoran You is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. You received his master’s degree in electrical and computer engineering from Rice University. Contact him at haoran.you@gatech.edu.
Haoran You is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. You received his master’s degree in electrical and computer engineering from Rice University. Contact him at haoran.you@gatech.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Yang Zhao is a postdoctoral researcher at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research interests include computer architecture and chip design. Zhao received her Ph.D. degree in electrical and computer engineering from Rice University. Contact her at yangkatie.zhao@gmail.com.
Yang Zhao is a postdoctoral researcher at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research interests include computer architecture and chip design. Zhao received her Ph.D. degree in electrical and computer engineering from Rice University. Contact her at yangkatie.zhao@gmail.com.View more
Georgia Institute of Technology, Atlanta, GA, USA
Cheng Wan is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include graph neural network and system design. Wan received his master’s degree in electrical and computer engineering from Rice University. Contact him at chwan@gatech.edu.
Cheng Wan is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include graph neural network and system design. Wan received his master’s degree in electrical and computer engineering from Rice University. Contact him at chwan@gatech.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Zhongzhi Yu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research focuses on machine learning. Yu received his master’s degree in electrical and computer engineering from Columbia University, New York. Contact him at zyu401@gatech.edu.
Zhongzhi Yu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research focuses on machine learning. Yu received his master’s degree in electrical and computer engineering from Columbia University, New York. Contact him at zyu401@gatech.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Yonggan Fu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Fu received his master’s degree in electrical and computer engineering from Rice University. Contact him at yfu314@gatech.edu.
Yonggan Fu is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Fu received his master’s degree in electrical and computer engineering from Rice University. Contact him at yfu314@gatech.edu.View more
Rice University, Houston, TX, USA
Jiayi Yuan is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Yuan received his bachelor’s degree in computer science from Tsinghua University. Contact him at jy101@rice.edu.
Jiayi Yuan is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Yuan received his bachelor’s degree in computer science from Tsinghua University. Contact him at jy101@rice.edu.View more
Rice University, Houston, TX, USA
Shang Wu is an M.S. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Wu received his master’s degree in computer science from Rice University. Contact him at sw99@rice.edu.
Shang Wu is an M.S. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Wu received his master’s degree in computer science from Rice University. Contact him at sw99@rice.edu.View more
Rice University, Houston, TX, USA
Shunyao Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Zhang received his master’s degree in computer science from Carnegie Mellon University. Contact him at sz74@rice.edu.
Shunyao Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on machine learning. Zhang received his master’s degree in computer science from Carnegie Mellon University. Contact him at sz74@rice.edu.View more
Rice University, Houston, TX, USA
Yongan Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on computer architecture. Zhang received his master’s degree in electrical and computer engineering from Rice University. Contact him at yz87@rice.edu.
Yongan Zhang is a Ph.D. student at Rice University, Houston, TX, 77005, USA. His research focuses on computer architecture. Zhang received his master’s degree in electrical and computer engineering from Rice University. Contact him at yz87@rice.edu.View more
Georgia Institute of Technology, Atlanta, GA, USA
Chaojian Li is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Li received his bachelor’s degree in electrical and computer engineering from Tsinghua University. Contact him at cli851@gatech.edu.
Chaojian Li is a Ph.D. student at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. His research interests include machine learning and computer architecture. Li received his bachelor’s degree in electrical and computer engineering from Tsinghua University. Contact him at cli851@gatech.edu.View more
Rice University, Houston, TX, USA
Vivek Boominathan is a postdoctoral researcher at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Boominathan received his Ph.D. degree in electrical and computer engineering from Rice University. Contact him at vivekb@rice.edu.
Vivek Boominathan is a postdoctoral researcher at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Boominathan received his Ph.D. degree in electrical and computer engineering from Rice University. Contact him at vivekb@rice.edu.View more
Rice University, Houston, TX, USA
Ashok Veeraraghavan is a professor at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Veeraraghavan received his Ph.D. degree in computational imaging from the University of Maryland. He is a Fellow of IEEE. Contact him at vashok@rice.edu.
Ashok Veeraraghavan is a professor at Rice University, Houston, TX, 77005, USA. His research focuses on computational imaging. Veeraraghavan received his Ph.D. degree in computational imaging from the University of Maryland. He is a Fellow of IEEE. Contact him at vashok@rice.edu.View more
Meta Reality Labs, Redmond, WA, USA
Ziyun Li is a research scientist at Meta Reality Labs, USA. His research interests include chip design and AR/VR. Li received his Ph.D. degree in electrical and computer engineering from the University of Michigan. Contact him at liziyun@fb.com.
Ziyun Li is a research scientist at Meta Reality Labs, USA. His research interests include chip design and AR/VR. Li received his Ph.D. degree in electrical and computer engineering from the University of Michigan. Contact him at liziyun@fb.com.View more
Georgia Institute of Technology, Atlanta, GA, USA
Yingyan (Celine) Lin is an associate professor at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research focuses on efficient machine learning via cross-layer innovations from algorithm to hardware architecture down to chip design. Lin received her Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana-Champaign. She is a Member of IEEE. Contact her at celine.lin@gatech.edu.
Yingyan (Celine) Lin is an associate professor at the Georgia Institute of Technology, Atlanta, GA, 30318, USA. Her research focuses on efficient machine learning via cross-layer innovations from algorithm to hardware architecture down to chip design. Lin received her Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana-Champaign. She is a Member of IEEE. Contact her at celine.lin@gatech.edu.View more
