Exploring the Essence of Prompt Tone: A Comparative Study of Large Language Model Security

Chatbots powered by Large Language Model(LLM) can be manipulated by malicious prompts, generating harmful content and biased responses which would raise security concerns. Growing dependence on chatbots demands robust security for ethical development and user trust, which makes the work relevant in today’s world. The motivation behind the work is to let the user have a safe experience with no negative responses being displayed while using the chatbot, which paved the way to arrive at the goal of developing a security filter that could be integrated into any LLM feature integrated application to mitigate the risk of having security vulnerabilities like prompt injection and jailbreaking, which could be achieved by converting malicious prompt into safer prompts by the method of eliminating negative sentiment phrases. The work focuses on building and implementing the security filters to popular in-production LLMs like Large Language Model Meta AI-2 (LLaMA2) and Generative Pre-trained Transformer – 3.5 turbo (GPT-3.5) to see how they handle against prompt injection and jailbreaking before and after the security filter being integrated. A large database of 200,000 prompts has been collected and preprocessed to train on a machine learning model using binary classification algorithm having 99.7% accuracy for classification of prompts into malicious or non-malicious and further checks are being done by breaking the prompt into smaller phrases and individually analyzing their compound sentiment score using Natural Language Toolkit (NLTK) Valence Aware Dictionary for Sentiment Reasoning (VADER) algorithm to detect and drop the negative sentiment phrases for the modification of the user prompt to eliminate the possibility of malicious prompt being passed to LLM. It is difficult to determine the sentiment of prompts in a detailed way and convert it into an efficient design that will perform well with models. Once this hurdle is overcome, chatbots will become even more reliable, trustworthy, and user-friendly tools, increasing the level of people’s trust in this technology, and establishing guidelines for how to use these resources in the most beneficial ways.