Neural network-based market clearing price prediction and confidence interval estimation with an improved extended Kalman filter method

Market clearing prices (MCPs) play an important role in a deregulated power market, and good MCP prediction and confidence interval (CI) estimation will help utilities and independent power producers submit effective bids with low risks. MCP prediction, however, is difficult, since MCP is a nonstationary process. Effective prediction, in principle, can be achieved by neural networks using extended Kalman filter (EKF) as an integrated adaptive learning and CI estimation method. EKF learning, however, is computationally expensive because it involves high dimensional matrix manipulations. This work presents a modified U-D factorization method within the decoupled EKF (DEKF) framework. The computational speed and numerical stability of this resulting DEKF-UD method are significantly improved as compared to standard EKF. Testing results for a classroom problem and New England MCP predictions show that this new method provides smaller CIs than what provided by the BP-Bayesian method developed by the authors. Testing also shows that our new method has faster convergence, provides more accurate predictions as compared to BP-Bayesian, and our DEKF-UD MCP predictions are comparable in quality to ISO New England's predictions.