Hierarchical Agglomerative Clustering and LSTM-based Load Prediction for Dynamic Spectrum Allocation

To improve spectrum efficiency without interfering with licensed users, reliable prediction of spectrum occupancy plays a pivotal role in a dynamic spectrum allocation (DSA) system. A reliable machine learning method capable of exploring the long-term correlation in the data is using a neural network with long short-term memory (LSTM). However, in the situation that there are multiple sensors in the network, how to effectively exploit the spatial correlation among these sensors' data for accurate spectrum prediction remains an open issue. Directly applying LSTM to multiple series may even reduce the prediction accuracy if some series are uncorrelated. In this article, we propose a method of clustering to aid in predicting multi-dimensional received power based on the hierarchical agglomerative clustering (HAC) model, which clusters the correlated series with high spearman's rank correlation coefficient (SRCC). Similar series are grouped into clusters and trained to predict independently using HAC. By ensuring uncorrelated series do not influence each other, the LSTM prediction accuracy is improved. A low-pass filter is used to remove high-frequency noise components and further reduce the prediction error. Experimental results show that our method significantly increases the prediction accuracy in all cases.