A Gesture Control Framework Targeting High-Resolution Video Wall Displays

This recent advances in display and user interface technologies create new opportunities for participatory urban planning. Whereas touch screens enable direct screen interactions for intuitive remodeling of urban simulations, high resolution video walls offer true-to-life or human-scale scenario visualizations. Multi-touch screen environments finally function as an interactive user interface for the entire urban simulation pipeline. However intuitive and preferably natural interactions for simple reconfiguration, overarching tool management or complex tool interaction remain a major challenge. For this reason we have implemented a gesture recognition framework that uses multimodal input devices to satisfy all gesture use cases and integrates a display management system for seamless interaction with our multi-screen infrastructure. To understand the implications of this novel approach, we conducted preliminary user trials with urban planning experts and technicians. The results indicated that the system provides a fluid and natural user experience for exploring and analyzing urban planning data.