Developing a 3D interface for sound corpus manipulation in virtual environments

This paper describes the implementation of 3D Corpus Navigator, a virtual musical instrument that can be used to interact and reconfigure a collection of sounds in 3D space, with applications in sound design, music composition, and audiovisual performance. Using different sound descriptors to analyze a sound corpus, the results are plotted as a 3D point cloud where each point represents a sound sample. By turning the user into a virtual ‘playhead’, a single user can interact with the 3D point cloud by means of virtual navigation, playing back the sound samples as the user flies through the point cloud. The user can interact via a game-pad or mouse and keyboard. The virtual musical instrument was implemented using Unreal Engine and the FluCoMa library in Cycling 74’s Max/MSP with bi-directional Open Sound Control communication. This paper discusses how this implementation advances prior work of similar systems, its advantages, and limitations. Future work and improvements are outlined for new iterations of the system at the end of this paper.