Statistical Evaluation of Classification Models for PCA Detection

A successful strategy to eavesdrop on the information exchange of legitimate users is by implementing a pilot contamination attack (PCA) during the process of channel estimation. Detection of such an intrusion can be accomplished at the BS and two promising methods that can be used for this purpose are 2-N-PSK and Shifted 2-N-PSK. As binary classification models, in this paper, the performance of both methods can be best evaluated through several commonly used statistical metrics, including statistical accuracy, precision and recall, specificity, fall-out and F
1
-score. Other useful information can be obtained from the Receiver Operating Characteristic (ROC) curve and the ROC Area Under Curve (AUC) that represent the trade-off between fall-out and recall. For the purpose of comparing the performance of 2-N-PSK and Shifted 2-N-PSK, in this paper each of the aforementioned metrics is extensively studied, and simulation results are provided and analyzed.