Markerless Avatar Animation with Forward Kinematics using Depth Sensor

Digital avatar is a graphical representation of a user in a virtual platform. To obtain an articulated and animated avatar, it is a labor-intensive job to perform pre- and post-processing for capturing body shapes and motions. The development of a markerless motion capture system has enabled the acquisition of body motion by eliminating marker attachments to the body. In this work, a markerless avatar rigging and animation pipeline is proposed. A deep learning-based rigging and skinning strategy is employed to automate the rigging process by estimating and aligning a virtual skeleton to the acquired 3D body shape and assigning the skin vertices to the skeleton. A depth sensor is used to capture the 3D pose by implementing a measurement-based approach. The performance measurement-based and estimation-based approaches are evaluated in term of motion capture. Subsequently, the motion of a user is streamed with the information of individual joint parameters, i.e. rotations in quaternion, to a web-based renderer for animation.