Preserving Hand-Drawn Qualities in Audiovisual Performance Through Sketch-Based Interaction

Live coding-the real-time procedural creation of audiovisual works-suggests opportunities to extend hand-drawn animation; however, existing live coding systems are incompatible with manual animation workflows. Manual input is not a primary datatype in existing live coding languages and live coding tools require using symbolic programming environments. We theorize that by applying direct manipulation to the domain of live coding, we can enable animators to create expressive mappings between hand-drawn animations and audio effects in real time. We present Megafauna, a sketch-based system for audiovisual performance, informed by interviews with professional animators. Megafauna supports the integrated generation and control of hand-drawn animation and audio sequences by enabling animators to directly sketch mapping functions between animation frames and sound oscillators. We demonstrate the expressive potential of Megafauna by reproducing animated compositions from procedural and manual domains and show how the system supports manual expression and exploration in live performance.