Virtual Experiment Platform Based on Chinese Speech Interaction

In this work, we present a module of Chinese speech interaction specifically for the 3-dimensinal (3D) virtual experimental platform. The use of speech interaction can improve the user's experience and be more efficient and natural in assembling the complex experimental devices. The assembly relationship graph and assembly sequence tree are used to constrain the assembly relationships and order of the experimental instruments. Based on the assembly tree structures and semantic information which is combined with the virtual scene, the assembling process can be guided by voice and text prompts, which clarifies the user's assembly operation and improve the interaction efficiency.