CoSSD - An end-to-end framework for multi-instance source separation and detection

We address the problem of separating classes of sound sources given 1) an audio mixture and 2) a conditional that represents the desired source to be separated. The proposed Conditional Source Separation and Detection (CoSSD) network operates on a defined set of classes during training and inference. We show that our model is versatile in that it can be repurposed for various separation tasks. We demonstrate its capability for conditional audio source separation on the NIGENS dataset of general sound events. We also show that, without modification, the model can perform speech and non-speech component sepa-ration using mixtures from the LibriSpeech and FSD50k datasets. Finally, a key feature of the proposed CoSSD is that it performs detection in addition to separation, making it a practical and unified solution for query-based audio analysis.