Protection of LLM Environment Using Prompt Security

The emergence of generative AI has also brought significant attention to large language models (LLMs). LLMs are a type of generative AI that can understand users' commands and respond accordingly by generating natural human language text. However, such advancements in LLM technology have also been accompanied by an increase in hostile attempts to use the technology for malicious attacks. For example, some may execute a jailbreak on an LLM, allowing it to generate malicious content, and others can use LLMs to generate vulnerability exploit codes, taking advantage of their powerful capabilities to attempt malicious attacks. Furthermore, there are increasing numbers of cases reported where, once inadvertently entered, personally identifiable information (PII) is used to train LLMs without consent. Preventing the malicious use of LLMs and their unauthorized use of PII requires thoroughly scanning their inputs and outputs. Specifically, LLMs' inputs and outputs need to be checked for the presence of PII, malicious codes, malicious URLs, and commands to execute prompt injections. This paper presents a comprehensive description of the LLM prompt detection system (LPDS) designed to detect the presence of PII, malicious codes, malicious URLs, and prompt injection commands in LLMs' inputs and outputs. The effectiveness of the LPDS in detecting LLMs' unauthorized use of PII and LLM-based malicious attacks is also assessed.