Analysis on Roles of DNNs in End-to-End Acoustic Scene Analysis Framework with Distributed Sound-to-Light Conversion Devices

We conduct an analysis on roles of deep neural networks (DNNs) in an end-to-end acoustic scene analysis frame-work with distributed sound-to-light conversion devices called Blinkies. Blinkies transmit sound information as the intensity of an onboard light-emitting diode (LED). A video camera can then easily collect acoustic information by capturing the LED intensities from multiple Blinkies distributed over a large area. In the end-to-end framework, both sound-to-light conversion and scene analysis processes are performed using two types of DNNs: an encoding network and a scene analysis network. These DNNs are optimized in an end-to-end manner for acoustic scene analysis. Although the efficacy of the end-to-end framework is already confirmed, it is unclear what role each network plays in the framework. In this paper, we examine the role of these networks through a simulation experiment using intermediate signal shuffling. Experimental results suggest that an encoding network does not output scene analysis results, but it achieves node-specific sound-to-light conversion that encodes spatial information of sounds. By using the node-dependent features obtained by encoding networks, the scene analysis network classifies scenes.