Closed-Form Pre-Training for Small-Sample Environmental Sound Recognition

This paper presents a framework for pre-training neural networks, namely closed-form pre-training, and we apply it to small-sample environmental sound recognition. Our main idea is to pre-train neural networks on a dataset automatically generated by some formulas, without any prior real-world recordings or manual annotation. Specifically, the proposed framework consists of two steps. First, an audio classification dataset is generated. Here, we propose three types of dataset definitions using colored noise and its extensions. Second, a network is pre-trained on the generated dataset. The obtained pre-trained network is particularly effective for fine-tuning with few examples because it helps optimization methods avoid falling into a premature local optimal solution. In experiments, we demonstrate the effectiveness of the proposed framework for small-sample environmental sound recognition on three datasets: ESC-10/50, and UrbanSound8K. We obtained performance improvement on all datasets with a small number of training samples.