QCon: A Q-Learning Based Congestion Control Scheme in Named Data Networking

In Named Data Networking (NDN), congestion control is essential for ensuring efficiency of data delivery. The present TCP/IP based end-to-end congestion control algorithms are inapplicable in NDN due to its multi-source and multipath transmission characteristics. Hop-by-hop congestion control is a distinctive feature in NDN that utilizes intermediate routers to control the congestion. However, the current congestion control schemes utilized in NDN are not up to the mark in controlling the congestion because of the varying network conditions of NDN. To overcome the limitations present in the current congestion control schemes, this paper proposes a novel Q-learning based hop-by-hop congestion control scheme named QCon. The proposed scheme utilizes congestion information (i.e., queue occupancy) from upstream nodes as a new field value in Data packet. The receiving router update the Q-values based on local congestion information and information received from upstream router and selects the path with the best Q-value for forwarding the Interest packets. The simulation results show that the proposed scheme outperform the similar hop-by-hop congestion control schemes in terms of packet loss ratio and throughput when encountered with congestion.