Constructing a model hierarchy with background knowledge for structural risk minimization: application to biological treatment of wastewater

This article introduces a novel approach to the issue of learning from empirical data coming from complex systems that are continuous, dynamic, highly nonlinear, and stochastic. The main feature of this approach is that it attempts to integrate the powerful statistical learning theoretic methods and the valuable background knowledge that one possesses about the system under study. The learning machines that have been used, up to now, for the implementation of Vapnik's inductive principle of structural risk minimization (IPSRM) are of the "black-box" type, such as artificial neural networks, ARMA models, or polynomial functions. These are generic models that contain absolutely no knowledge about the problem at hand. They are used to approximate the behavior of any system and are prodigal in their requirements of training data. In addition, the conditions that underlie the theory of statistical learning would not hold true when these "black-box" models are used to describe highly complex systems. In this paper, it is argued that the use of a learning machine whose structure is developed on the basis of the physical mechanisms of the system under study is more advantageous. Such a machine will indeed be specific to the problem at hand and will require many less data points for training than their black-box counterparts. Furthermore, because this machine contains background knowledge about the system, it will provide better approximations of the various dynamic modes of this system and will, therefore, satisfy some of the prerequisites that are needed for meeting the conditions of statistical learning theory (SLT). This paper shows how to develop such a mechanistically based learning machine (i.e., a machine that contains background knowledge) for the case of biological wastewater treatment systems. Fuzzy logic concepts, combined with the results of the research in the area of wastewater engineering, will be utilized to construct such a machine. This machine has a hierarchical property and can, therefore, be used to implement the IPSRM.