Security Dataset Augmentation Invariance and Distribution Independence

Security-focused dataset augmentations, e.g., pen-testing a firewall via fuzzing, differ significantly from other domains. The number of available attack samples is severely limited and malicious actors often modify requests to appear different while achieving the same effect. These properties violate common distribution assumptions needed to accurately train machine learning (ML) models. Conversely, generating high-quality labels for these datasets is unlike similar areas, such as natural language processing (NLP). Security categorizations are determined by machine behavior based on defensive detection and request logic preservation. This observation allows augmentation methods beyond the typical distribution-preserving methods. The results are of interest both offensively and defensively to subvert existing protections and enhance detections respectively. This paper explores generating augmentations for a security focused SQL dataset that preserves the intended function using the described automated corpus labeling. Three different augmentation techniques are tested, along with their combinations. The methods are chosen to represent statistical mutations and adversarial scenarios. Evaluation is performed with a downstream classification task for categorizing queries. The results show that in all cases augmentation provides benefits over the initial dataset. This is especially pronounced in augmentation methods that represent extended attack logic. The techniques presented can be added to any form of network defensive model to increase the effectiveness of an initial small data corpora.