By Topic

3D Auditory Scene Visualizer with Face Tracking: Design and Implementation for Auditory Awareness Compensation

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

6 Author(s)
Yuji Kubota ; Grad. Sch. of Inf., Kyoto Univ., Kyoto, Japan ; Shun Shiramatsu ; Masatoshi Yoshida ; Kazunori Komatani
more authors

This paper presents the design and implementation of 3D Auditory Scene Visualizer based on the visual information seeking mantra, ``overview first, zoom and filter, then details on demand''. The machine audition system called HARK captures 3D sounds with a microphone array.The natural language processing called SalienceGraph visualizes topic transition by using discourse salience. The 3D visualizer implemented in Java 3D displays topic transition and each sound stream as a beam originating from the microphones (overview mode), shows temporal snapshots with/without specifying focusing areas (zoom-and-filter mode), and shows detailed information about a particular sound stream (details-on-demand mode). This three-mode visualization will give the user auditory awareness enhanced by HARK and SalienceGraph. In addition, a face-tracking system automatically determines the user's intention by tracking the user's face. The resulting system will enable users to manage and browse auditory scene files effectively, so it should acceleration and support the information explosion to compensate the lack of auditory awareness.

Published in:

Universal Communication, 2008. ISUC '08. Second International Symposium on

Date of Conference:

15-16 Dec. 2008