TFI-AMR: A Transformer-based Forest-Inspired Approach for Automatic Modulation Recognition in Wireless Communication

Automatic Modulation Recognition (AMR) allows efficient enhancement of wireless communication, especially when prior information about the modulation scheme is absent. With the advent of deep learning, AMR has seen significant advancements, particularly with the introduction of the Transformer architecture, known for its self-attention mechanism. Inspired by the adaptive and interconnected nature of forests, we introduce the Transformer-based Forest-Inspired Approach for AMR (TFI-AMR). This study aims to harness the self-attention mechanism of the Transformer model, integrating it with concepts derived from the forest ecosystem to develop a robust and adaptive AMR system. The TFI-AMR model is structured hierarchically, mirroring the layers of a forest – from the root to the leaves. Each layer focuses on different granularity levels of signal features, from global to highly specific. The adaptive growth mechanism, inspired by the growth rings of trees, represents the learning epochs of the model. Pruning, akin to trees shedding branches, optimizes the model by removing non-essential attention heads or layers. The multi-head attention mechanism is visualized as different trees or species in a forest, each focusing on diverse aspects of the signal. Positional encoding introduces adaptability, reminiscent of seasonal changes in a forest. A unique “forest fire” mechanism occasionally resets parts of the model, ensuring variability and preventing overfitting. The experimental evaluation of the TFI-AMR model demonstrated high adaptability to new modulation schemes, robust performance due to the multi-head attention and “forest fire” mechanisms, and enhanced explainability through its hierarchical structure and attention maps.