An Incremental Decoding Scheme for Optimal Real-Time Control of Markovian Systems

Extremely low latency communications has attracted considerable attention because of its potential in emerging real-time applications such as autonomous driving, smart grids, and Industrial Internet of Things (IIoT). Incremental decoding, which allows a remote controller to make real-time decisions without waiting for the whole packet or codeword to be decoded, may significantly reduce the latency-induced cost, especially when the bandwidth is very limited. In this paper, we are interested in the incremental decoding for the real-time control of a Markovian system. In contrast to the conventional Markov decision process, in which the decision maker receives the state information immediately, we consider a remote control scenario in which the transmission delay of its remote state information is non-negligible. To maximize the average reward in such Markovian systems, we conceive a joint incremental decoding and Markov decision making policy. In particular, the instantaneous codebook is determined by the controller's local state, which is also known by the sensor, while the controller can make its decision in real time based on partially received packet. Furthermore, we demonstrate the policy's potential by applying it into automated highway driving scenario, the performance gain of which has been revealed by numerical results.