An Improved Spray and Wait Algorithm Based on Q-learning in Delay Tolerant Network

Unlike traditional networks, Delay Tolerant Network (DTN) does not have a stable end-to-end connection, so it adopts the store-carry-forward transmission mode. In DTN routing, the context information of nodes and messages, such as the social attributes between nodes, the buffer occupancy of nodes and the hop count of messages, is very important for the design of routing algorithms. Considering the above factors, this paper proposes an improved spray and wait routing algorithm based on Q-learning (ANSAW-Q). Firstly, this paper proposes a social circle construction method to characterize the similarity of social circles between nodes. Secondly, exploiting the distinctive features of the DTN routing process, we introduce connection and transmission reward functions that leverage context information of nodes and messages to update the Q-value. Finally, an adaptive method for initial message replication is introduced based on the relationship between the source and the destination. In the routing process, the node selects the appropriate relay node and manages the buffer according to the Q value. Simulation results show that ANSAW-Q can improve the message delivery ratio and reduce the network load. Q-learning can effectively improve spray and wait algorithm.