mmAcoustic: Full-Field Sound Source Localization, Identification, and Area-Selectable Sound Recovery via Millimeter-Wave Vibration Monitoring

Full-field sound source localization, identification, and area-selectable sound acquisition are highly desirable for a wide spectrum of applications in acoustic sensing field. However, traditional microphone and emerging laser or visual microphone techniques remain fundamental limitations. In this article, we propose a millimeter-wave full-field vibration monitoring-based acoustic sensing approach, creating a unique mmAcoustic that enables the synchronous perception of multiple sound sources with range–angle joint localization, automatic recognition, and area-selectable sound recovery, which can fundamentally get rid of the key issues of multisound signal aliasing, reverberation, and noise interference in a complex acoustic environment. Considering sound signals are fundamentally produced by vibrations, in mmAcoustic, the tiny vibration signals of full-field targets are first accurately measured and further processed to actively perceive sound sources. To this end, we establish the method of feature-driven sound source identification with extracting nine typical features and following with classification of four main types. In addition, the desirable area-selectable sound pickup can be achieved by signal reconstruction from the measured vibration signal of the target sound source in a certain range–angle joint bin, following with broadband signal recovery with deep learning-based postprocessing. Experimental results show that the sound source identification accuracy can be achieved with almost 100%, and the desired area-selectable sound acquisition can be conveniently achieved along with high-quality sound signal recovery.