Neural Layer Design of Pragmatic Communication: Inspiration of the Structure of Natural Languages

In complex environments, cooperative agents need to convey a large amount of cognitive and decision-making information, and the huge number of parameters of neural networks increases the computational burden, and it is also very difficult to traverse all possibilities. In this paper, we enhance the architecture of the “Glue Neural Layers” (GNLs) to improve the efficiency of inter-machine communication without changing the information transfer size. We found that using ‘word’ units for communication is more effective than using ‘letters’, but too many ‘words’ can lead to a large vocabulary and affect training. Therefore, inspired by natural language, we propose to use ‘sentence’ units to reduce dictionary thickness and improve communication. Through extensive experiments, we identify the optimal balance between ‘vocabulary’ and ‘sentence length’ for different environmental complexities and verify that using ‘sentences’ is better than using ‘words’ and that there exists an optimal ‘sentence length’ for a specific task. This study sheds light on the relationship between neural architecture and language architecture.