Experiments in using structured musical sound, synthesised speech and environmental stimuli to communicate information: is there a case for integration and synergy?

This paper describes three sets of experiments in auditory information processing using structured musical sound, synthesised speech and environmental stimuli or special effects. The first experiment examines auditory information processing of small sequences of rhythmic musical tones. The second experiment examines auditory information processing of some environmental sounds. The third experiment examines auditory information processing when sound and synthesised speech are simultaneously presented. The results of this investigation aim to help multimedia and user interface developers to design auditory messages that incorporate structured musical stimuli, synthesised speech and environmental sounds or special effects. These type of auditory messages can complement visual displays or communicate information on their own in auditory interfaces. The results of these initial experiments indicate a prima facie case for integrating various types of auditory stimuli in one single message. The simultaneous presentation of sound and speech was easily recognised and interpreted. The paper concludes with some initial practical guidelines for designers and a discussion of further work.