Multimedia e-mail data browsing: the synergistic use of various forms of auditory stimuli

This paper describes eight experiments that investigate the possibility of using compositional sounds, stereophony, structured musical stimuli (earcons), environmental sounds (auditory icons) and speech to communicate e-mail categories and other e-mail related information such as attachment, priority, and subject of e-mail messages. The results of this investigation contribute to reduce visual complexity, which is often encountered during graphical browsing of e-mail data. E-mail categories were communicated using sequences of 10, 4 and 2 seconds of compositional sounds (4 experiments with each duration) and by combining the communication of e-mail category with e-mail status information using left, centre and right stereophonic positions (1 experiment), the presence or absence of an attachment using an environmental sound (1 experiment), priority using tones (1 experiment) and subject using speech (1 experiment). The results of these experiments demonstrate that short sequences (typically 2 to 4 seconds) of compositional sounds were sufficient for users to recognise the category of a particular e-mail. They also demonstrate that status information of an e-mail (unread, read, replied and forwarded), priority (low, medium and high), the presence or absence of an attachment and subject were also communicated successfully using additional auditory stimuli in the same auditory message that communicated the e-mail category. Thus the approach of combining different types of auditory stimuli to communicate larger volumes of data demonstrated that it has the potential to reduce complexity in graphical browsing by communicating some information aurally. Issues of perceptual context and synchronisation of media have also been identified.