LSTM Hyper-Parameter Selection for Malware Detection: Interaction Effects and Hierarchical Selection Approach

Long-Short-Term-Memory (LSTM) networks have shown great promise in artificial intelligence (AI) based language modeling. Recently, LSTM networks have also become popular for designing an AI-based Intrusion Detection Systems (IDS). However, its applicability in IDS is studied largely in the default settings as used in language models. Whereas security applications offer distinct conditions and hence warrant careful consideration while applying such recurrent networks. Therefore, we conducted one of the most exhaustive work on LSTM hyperparameters for IDS and experimented with 150 LSTM configurations to determine its hyper-parameters' relative-importance, interaction-effects, and optimal selection-approach for designing an IDS. We conducted multiple analysis of the results of these experiments and empirically controlled for the interaction effects of different hyper-parameters covariate level. We found that for security applications, especially for designing an IDS, neither similar relative importance as applicable to language models is valid, nor is the standard linear method for hyper-parameter selection ideal. We ascertained that interaction effect plays a crucial role in determining the relative importance of hyperparameters. We also discovered that after controlling for the interaction-effect, the correct relative importance for LSTMs for an IDS are batch-size, followed-by dropout ratio and padding. The findings are significant because when LSTM were first used for language models, the focus had mostly been on increasing the number of layers to enhance performance.