Variants of Recursive Consequent Parameters Learning in Evolving Neuro-Fuzzy Systems

A wide variety of evolving (neuro-)fuzzy systems (E(N)FS) approaches have been proposed during the last 10 to 15 years in order to handle (fast and real-time) data stream mining and modeling processes by dynamically updating the rule structure and antecedents. The current denominator in the update of the consequent parameters is the usage of the recursive (fuzzily weighted) least squares estimator (R(FW)LS), as being applied in almost all E(N)FS approaches. In this paper, we propose and examine alternative variants for consequent parameter updates, namely multi-innovation RFWLS, recursive corr-entropy and especially recursive weighted total least squares. Multi-innovation RLS guarantees more stability in the update, whenever structural changes (i.e. changes in the antecedents) in the E(N)FS are performed, as the rule membership degrees on (a portion of) past samples are actualized before and properly integrated in each update step. Recursive corr-entropy addresses the problematic of outliers by down-weighing the influence of (atypically) higher errors in the parameter updates. Recursive weighted total least squares takes into account also a possible noise level in the input variables (and not solely in the target variable as in RFWLS). The approaches are compared with standard RFWLS i.) on three data stream regression problems from practical applications, affected by (more or less significant) noise levels and one embedding a known drift, and ii.) on a realworld time-series based forecasting problem, also affected by noise. The results based on accumulated prediction error trends over time indicate that RFWLS can be largely outperformed by the proposed alternative variants.