VirtualIMU: Generating Virtual Wearable Inertial Data from Video for Deep Learning Applications

In the era of deep learning, accessibility to a large amount of wearable Inertial Measurement Unit (IMU) data plays a crucial role in various biomedical and health applications. However, collection of ‘big’ IMU data is extremely challenging due to its cost and time requirements. To address this, researchers have explored using publicly available videos, such as those on YouTube, to extract human skeletal models and synthesize IMU data. However, existing methods for converting skeletal models to virtual IMU data are oversimplified and lack systematic data augmentation capabilities. In this study, we propose a systematic approach to synthesize realistic and diverse IMU data, including three-axis accelerometer and gyroscope measurements, from video-based skeleton representations. Through experiments involving seven healthy individuals, we demonstrate that our method can accurately synthesize accelerometer and gyroscope data with a normalized root mean square error of 14.4 % and 16.0 %, respectively. Furthermore, we qualitatively evaluate the algorithm’s ability to generate a large volume of diverse IMU data. Our findings affirm the potential of obtaining diverse synthetic IMU data from videos, offering a promising solution to reduce the costs associated with collecting IMU data in deep learning-based applications.