$QRTT$ : Stateful Round Trip Time Estimation for Wireless Embedded Systems Using $Q$-Learning

Wireless embedded systems such as sensor nodes and smartphones highlight the importance of reliable data transmission in their advanced applications. Such reliable transmission frequently exhibits a performance below expectation due to inherent resource constraints of the systems. Accurate estimation of round trip time, an important parameter of reliable data transmission, has the potential to significantly improve the performance withstanding the resource constraints. However, contemporary round trip time estimation schemes cannot significantly improve the performance using their stateless estimation schemes as two different states of transmission, success and failure, are equally prominent over wireless mediums. To address this key fact, we propose a novel stateful round trip time estimation scheme for wireless embedded systems, incorporating a light-weight artificial intelligence method. We apply the proposed scheme in a retransmission timeout mechanism of TCP and compare the efficacy of the scheme against that of contemporary estimation schemes. Exhaustive simulation and a testbed experiment reveal that our scheme significantly improves network throughput and average energy per bit compared to other schemes.