Hand and finger registration for multi-touch joysticks on software-based operator control units

Robot control typically requires many physical joysticks, buttons, and switches. Taking inspiration from video game controllers, we have created a Dynamically Resizing, Ergonomic, and Multi-touch (DREAM) controller to allow for the development of a software-based operator control unit (SoftOCU). The DREAM Controller is created wherever a person places his or her hand; thus we needed to develop an algorithm for accurate hand and finger registration. Tested with a set of 405 hands from 62 users, our algorithm correctly identified 97% of the hands.