Examining traffic microstructures to improve model development

We demonstrate how machine-learning-based network intrusion detection models can be validated and developed by probing models using traffic with specifically controlled microstructures. We show our methodology by probing two published state-of-the-art models to find classification flaws and and understand misbehaviour. These models fail for input traffic with particular characteristics such as retransmissions or overly dispersed flow interarrival times. After we make simple corresponding model corrections, detection rates already improve between 2 -4%. We believe this shows promise for using tailored data with controllable and labelled characteristics to effectively improve model development in NID, a practice that helped model development significantly in several other areas of machinelearning.