Assessment of Extreme Communication Environment With Ultralow SNR: A Benchmark

Accurate estimation of subjective assessment plays an essential role in not only speech quality perception, but also communication environment assessment. Traditional speech quality perception is almost always with an environment, in which the content of speech can be heard clearly. Unlike speech quality assessment, in extreme communication environment with ultralow SNR, like short wave channel with active jamming, speech intelligibility is impaired. Under this condition, subjective assessment as absolute category ranking (ACR) whose scores are made by experienced staffs, and many objective measurements cannot handle this situation. In this paper, we propose information damage level (IDL) to replace subjective ACR as subjective assessment. IDL is the average of scores which is marked on the subjective recognition rate. Under extreme communication environment, it can effectively avoid excessive differences from people to people to some extent. We also provide a new dataset collected in an environment with active jamming, whose speech files are recoded under three different environments, named as indoor simulation environment (EN1), outdoor simulation environment (EN2), and outdoor real environment (EN3). We also benchmark a novel open framework on random forest for direct predicting subjective assessment by combining all possible objective measurements. Experiment results prove the effectiveness of our open framework together with our dataset.