Network Intrusion Detection with Improved Feature Representation

This paper presents new feature representation methods for the network intrusion detection. Like conventional work, our method is based on neural networks. However, rather than focusing on the network architecture search, we improve the performance by developing effective feature representation methods. First, we apply the embedding method to categorical features in the network data. Although embedding has been commonly used in natural language processing and recommen-dation systems, categorical features in network problems were often ignored or simply used in a one-hot-encoding vector form. By applying the embedding method to categorical features, we can effectively exploit them. Second, we apply a robust scaler to numerical features. Numerical features are concentrated in a few clusters with a small portion of outliers, and we can effectively remove outliers with a robust scaler. Finally, we augment feature vectors with categorical representations of numerical values. These categorical representations (e.g., high, medium, and low) help to discover simple logical rules and facilitate the intrusion detection. We have applied our method to two scenarios: a normal/attack classification and an unsupervised learning of anomaly detection. Experimental results have shown that the proposed method outperforms conventional methods on public benchmark datasets.