A data-driven approach for multivariate contextualized anomaly detection: Industry use case

Anomaly detection is the process of discovering some anomalous behaviour in the real-time operation of a system. It is a difficult task, since in a general case (multivariate anomaly detection) an anomaly can be related to the behaviour of several parameters which are not necessarily behaving anomalously per se, but their (complex) relation is anomalous (not usual/normal). This implies the need for a very efficient modeling of the normal behaviour in order to know what should be treated as anomalous/outlier/unusual. Consequently, classical model-driven approaches, due to their focus on the selected parameters for creating models, are not able to model the behaviour of the whole system. This is why data-driven approaches for anomaly detection are getting even more important for the industry use cases where hundreds (thousands) of parameters should be taken into account. However, current approaches are usually focused on the univariate anomaly detection (or some variations of it), so without observing the entire space of relations since the computation is very complex. In this paper we present a novel approach for the multivariate anomaly detection that is based on modeling and managing the streams of variations in a multidimensional space. The main advantage of this approach is the possibility to observe the relations between variations in a large set of parameters and create clusters of “normal/usual” variations. In order to ensure scaling, which is one of the most challenging requirements, the approach is based on the usage of the big data technologies for realizing data analytics tasks/calculations. The approach is realized as a part of D2Lab (Data Diagnostics Laboratory) framework and has been applied in several industrial use cases. In this paper we present an interesting usage for the anomaly detection in the process of functional testing of home appliances (in particular case refrigerators) after manufacturing/assembling process. It has been done for a big vendor (Whirlpool), who expects huge saving in testing and improved customer satisfaction from this approach.