HSM:Hand Surface Menu Solution for Interaction in VR

The menu interaction methods in VR, such as floating menus, are still considered unnatural. A solution is proposed in this paper where menus are tightly attached to the user’s palm. Firstly, use UV mapping technology to attach the menu to the palm. Secondly, optimal layout parameters for the hand surface menu were determined through user experiments. Finally, a dataset was built to train the interaction error detection model, aiming to improve interaction success rates under various hand poses in VR. Experimental results demonstrate that compared to other VR menu methods, HSM reduces the average interaction error rate by 30.1% and shortens the average completion time by 12.4% under the optimal hand pose.