Split Learning-based Sound Event Detection in Energy-Constrained Sensor Devices

Sound event detection (SED) using lightweight sensor device has recently gained attention as a practical means to capture context and activities especially in domestic environments. However, SED applications running on sensor device are severely constrained by device’s energy capacity. One solution is to offload a portion of inference to server for reducing runtime complexity, i.e., energy consumption, of sensor device. Offloading should consider the trade-off between computation and data transmission costs adequately; more computation on sensor device reduces data to be transmitted and vice versa. To address this challenge, we propose SEDAC (Sound Event Detection with Attention-based audio Compression), a novel technique for split learning in SED that compresses data from sensor device to offload less data. SEDAC compresses the input of SED models, or Mel spectrograms, with minimal computation in sensor device. Rather than directly compressing the input, SEDAC achieves data compression by selectively capturing the key parts of sound events using an attention mechanism. The scheme also modifies an existing loss function and employs knowledge distillation to mitigate potential loss of SED accuracy due to data compression. Our evaluation shows that SEDAC outperforms the state-of-the-art data compressive split learning schemes, up to about 30%. Furthermore, our real-world deployment demonstrates that sensor devices with SEDAC successfully operate with minimal energy and memory overhead.