Simultaneous neural and movement recording in large-scale immersive virtual environments

Virtual reality (VR) allows precise control and manipulation of rich, dynamic stimuli, which when coupled with on-line motion capture and neural monitoring can provide a powerful means both of understanding brain behavioral relations in the high dimensional world, and of assessing and treating a variety of neural disorders. Here we present a system that combines state-of-the-art, fully immersive, 3D, multi-modal VR with temporally aligned electro-physiological recordings. The VR system is dynamic and interactive across visual, auditory, and haptic interactions, providing sight, sound, touch, and force. The overall end-to-end latency between real movement and its simulated movement in the VR is approximately 40ms. Spatial precision of the various devices is on the order of millimeters. The temporal alignment with the neural recordings is accurate to within approximately 1ms. This powerful combination of systems opens up a new window into brain-behavioral relations and a new means of assessment and rehabilitation of individuals with motor and other disorders.