MoshViz: A Detail+Overview Approach to Visualize Music Elements

A music piece contains a large amount of information represented as a series of instructions corresponding to notes that must be played at specific times. These simple notes are combined to form complex harmonic structures that can be difficult to identify and analyze. Due to its simplicity and straightforward interpretation, music sheets and piano rolls have been the visual metaphor employed by most music visualization tools to support interpretation. Albeit it can represent all necessary elements to perform a music piece, these metaphors do not explicitly show many of the patterns and structures inherent to music arrangements, such as rhythm progression and harmonic interactions, needing users to create a mental model of them. Moreover, comparing different pieces and visualizing how a particular instrument track relates to the others is an issue not only for music sheet-based techniques, but also for most existing music visualization methods. In this paper, we present a novel visualization framework, called Music Overview, Stability, and Harmony Visualization (MoshViz), which facilitates the visualization and understanding of music renditions, focusing mainly on the visual analysis of specific musical instruments. Our approach creates a high-level model of music data and highlights structures of interest, enabling a detail+overview visualization to assist users in the task of identifying harmonic and melodic patterns. The usefulness and representativeness of MoshViz are confirmed by a set of user tests which demonstrate that the proposed visual metaphor matches, with a high degree of accuracy, the mental model of different users regarding the recognizable patterns of sounds.