Adaptive CNN-Transformer Fusion for Discriminative Fuzzy $C$-Means Clustering in Multivariate Forecasting

With the rapid advancement of artificial intelligence, an increasing number of researchers have begun to integrate deep learning with machine learning to propose a variety of meaningful prediction models. However, determining the optimal number of convolutional and pooling layers in a convolutional neural network (CNN) remains a challenging problem. In addition, the randomness in the query, key, and value vectors of the self-attention mechanism in Transformers can affect their predictive performance. Furthermore, inaccuracies in the calculation of the membership matrix in fuzzy $C$-means clustering (FCM) can impact clustering effectiveness. To address these issues, this article introduces an adaptive CNN-Transformer multivariate prediction model based on discriminative fuzzy $C$-means clustering (DFCM), referred to as DFCM-A-CNN-Transformer. First, by considering the membership degree of similar samples belonging to the same cluster, DFCM is designed to resolve inaccuracies in the computation of the membership degree matrix. Next, a system is developed to simultaneously determine the value of $k$ and the number of clusters in the $k$-nearest neighbor algorithm, thereby completing feature selection. Subsequently, the number of convolutional and pooling layers is adaptively determined based on data characteristics and dynamic stopping conditions. The query, key, and value vectors of the self-attention mechanism in the Transformer are then determined based on the output of the CNN. Finally, the superiority and effectiveness of the proposed model are validated through comparative experiments with six other models on eight datasets.