Optimal Sensor Scheduling for Remote State Estimation with Partial Channel Observation

Dear Editor, This letter investigates the optimal transmission scheduling problem in remote state estimation systems over an unknown wireless channel. We propose a partially observable Markov decision Process (POMDP) framework to model the sensor scheduling problem. By truncating and simplifying the POMDP problem, we have established the properties of the optimal solution under the POMDP model, through a fixed-point contraction method, and have shown that the threshold structure of the POMDP solution is not easily attainable. Subsequently, we obtained a suboptimal solution via Q-learning. Numerical simulations are used to demonstrate the efficacy of the proposed Q-learning approach.