Deep-Learning Enabled Assessment of Neurocognitive Performance in Object Following in Mixed Reality

The objective of this article is to develop a deep learning model to construct a comprehensive, machine-learnable representation of human performance that spans visual, cognitive, and motor-control abilities associated with an object-following task in mixed reality (MR). Compared to direct observations by trained clinical staffs, which is the current standard for clinical diagnosis, a deep learning approach is expected to detect subtle signs of neurocognitive abilities and/or impairment. If successful, the resultant representation will bring a new opportunity to be shared and communicated with humans, a first step to collaborative workflows between clinical staffs and artificial intelligence (AI) specialists for diagnosis.