Machine Learning-Based Run-Time DevSecOps: ChatGPT Against Traditional Approach

DevSecOps paradigm strives to provide the extension of the already approved DevOps practice by tackling the issues related to security in both design-and run-time. This paper focuses on aspects of run-time security and proposes a server log analysis-based machine learning-empowered method. The goal is to detect suspicious activity based on traffic-related information. For this purpose, we propose a novel dialogue-alike approach relying on ChatGPT in Python, where context represents labelled data, while the questions themselves contain the log records which are about to be evaluated if they represent suspicious activity or not. On the otter side, a traditional machine learning approach treating the problem as classification using Weka API in Java is also presented, for comparison. The evaluation was performed on publicly available server log dataset from Kaggle. According to the achieved results, ChatGPT-based approach shows potential to be used for run-time DevSecOps scenarios, but still has many limitations and drawbacks (limited context, processing time, costs, lower accuracy) while it is still not as effective as traditional, already approved classification techniques.