Deep Anomaly Detection for Network Traffic

Canonical anomaly detection has been achieved through various means ranging from statistical tests and clustering methods to categorical decision-making and rule-based systems. Each method has its own pros and cons; however, many depend on assumptions. These assumptions can be model driven, such as assuming white Gaussian inputs, or method driven such as linear regression. In any case, assumptions are being made either about the structure of the data or its relationship with other random variables.This work presents a deep learning methodology for anomaly detection, a sampling technique for large data sets, and feature importance analysis. The anomaly detection technique uses an ensemble of learners to predict relationships between benign features and characterizes deviations from these patterns as "surprisal" scores. This method identifies malicious network traffic without previous attack behavior knowledge and is applied to data from the Canadian Institute for Cybersecurity.