The Molecular Control Toolkit: Controlling 3D molecular graphics via gesture and voice

Three-dimensional (3D) molecular graphic systems are widely used in the life sciences, both for research and communication. These systems need to enable a rich set of 3D operations, including three-axis rotation and translation, selection of parts of macromolecules, and the ability to redefine the center of rotation. As a result, graphical interfaces for these systems typically require users to learn complex keyboard and mouse combinations. This can be a significant barrier for new or occasional users, and even for experts, precise control of 3D molecular structures can be challenging. To help address these challenges, we developed the Molecular Control Toolkit to support multiple consumer gesture and voice recognition devices, and provide an API that allows adaption to multiple molecular graphics systems. The toolkit allows intuitive control, almost as if users are directly manipulating 3D objects in their hands. We applied the toolkit to the Kinect and Leap Motion devices, and to the Aquaria molecular graphics system. We did a pilot user study with 18 life scientists to test the resulting system in different scenarios. Overall, users gave quite favorable ratings to using the Kinect and Leap Motion gesture devices to control molecular graphics, even though these devices initially proved less efficient for common 3D control tasks, compared to the more familiar mouse/keyboard. To