How to Tell a Bad Filter Through Monte Carlo Simulations

In this note, we propose one particular method to address the issue of how to numerically evaluate nonlinear filtering algorithms and/or their software implementations, through Monte Carlo simulations. We introduce a quantitative performance indicator whose computation can be automated and does not depend on any specific definition of point estimate. The method is based on conditional probability integral transform and maximum deviation of an empirical cumulative distribution function from a uniform distribution. The usefulness of such an indicator is illustrated through an example.