MOLESTRA: A Multi-Task Learning Approach for Real-Time Big Data Analytics

Modern critical infrastructures are characterized by a high degree of complexity, in terms of vulnerabilities, threats, and interdependencies that characterize them. The possible causes of a digital assault or occurrence of a digital attack are not simple to identify, as they may be due to a chain of seemingly insignificant incidents, the combination of which provokes the occurrence of scalar effects on multiple levels. Similarly, the digital explosion of technologies related to the critical infrastructure and the technical characteristics of their subsystems entails the continuous production of a huge amount of data from heterogeneous sources, requiring the adoption of intelligent techniques for critical analysis and optimal decision making. In many applications (e.g. network traffic monitoring) data is received at a high frequency over time. Thus, it is not possible to store all historical samples, which implies that they should be processed in real time and that it may not be possible to re-review old samples (one-pass constraint). We should consider the importance of protecting critical infrastructure, combined with the fact that many of these systems are cyber-attack targets, but they cannot easily be disconnected from their layout as this could lead to generalized operational problems. This research paper proposes a Multi-Task Learning model for Real-Time & Large-Scale Data Analytics, towards the Cyber protection of Critical Infrastructure. More specifically, it suggests the Multi Overlap LEarning STReaming Analytics (MOLESTRA) which is a standardization of the "Kappa" architecture. The aim is the analysis of large data sets where the tasks are executed in an overlapping manner. This is done to ensure the utilization of the cognitive or learning relationships among the data flows. The proposed architecture uses the k-NN Classifier with Self Adjusting Memory (k-NN SAM). MOLESTRA, provides a clear and effective way to separate the short-term from the long-term memory. In this way the temporal intervals between the transfer of knowledge from one memory to the other and vice versa are differentiated.