Temporal Difference Learning Model for TCP End-To-End Congestion Control in Heterogeneous Wireless Networks

Conventional TCP end-to-end Congestion Control approaches cannot be applied directly to heterogeneous wireless networks as TCP is unable to distinguish the reason for the packet loss. The high error rates in wireless networks are due to interference or frame collisions caused by multiple simultaneous transmissions resulting in throughput degradation. There exist several ML approaches towards congestion control but neither the supervised nor the unsupervised learning techniques are suitable for learning the optimal policy. Therefore, a model is to be developed that predicts the optimal congestion window by interacting with the environment dynamically. To address these challenges, we propose a reinforcement learning model to dynamically adjust the congestion window using the Actor-Critic method and Temporal Difference learning. From the experiments, it is evident that the proposed learning model achieves 40% more throughput than the existing techniques while maintaining low transmission latency.

I. Introduction

A Heterogeneous wireless network is a combination of several wireless networks which includes cellular networks, ad-hoc networks and wireless LAN's, connected to the Internet. Heterogeneous wireless networks provide a diverse wireless network access. Improving the communication throughput, extending the service coverage of the existing cellular and WLAN infrastructure, reducing packet transmission latency based on availability of different types of radio network in the roaming area are some of the advantages of Heterogeneous wireless networks.