Over The Air Performance of Deep Learning for Modulation Classification across Channel Conditions

Deep learning (DL) models used for modulation classification are mostly trained on simulated data. Their performance drops significantly on real test data, due to disparity in probability distributions between simulated and real data. The eventual goal is building a DL model classifying modulation type accurately on real data. This work empirically studies the performance impact due to disparity in probability distributions between training and test data. We borrow best performing deep learning models from literature for our analysis. Models are tested on data belonging to channel conditions they were trained on and otherwise. Software defined radios (SDR) collect training and test data under channel conditions of additive white Gaussian noise, line-of-sight (LOS) and non-line-of-sight (NLOS). Convolutional neural network (CNN) and Residual neural network (ResNet) architectures are used. Test accuracies of the models are compared across model architectures, channel conditions, modulation types and SNR. Performance results of DL models on real data, are presented for wide set of scenarios. Dataset is available for download and can be used for evaluating deep learning models.