Assessing Bias in Large Language Models: A Comparative Study of ChatGPT, Gemini, and Claude

As Large Language Models (LLMs) become more integrated into various applications, their potential to display and amplify biases is a significant concern. This study evaluates the social and gender bias in ChatGPT, Gemini, and Claude AI. Unlike previous studies using pre-existing datasets, we used handcrafted questions to reveal biases more effectively. Basic simple questions provided an unbiased answer, however complex biased questions highlighted a few concerns. ChatGPT had a bias all the time while Gemini had a bias sometimes and the rest of the time it did not, or if it did the bias was different, while Claude AI was mostly unbiased. Thus, this study shows that there is variation in levels of bias in different AI models and thus advocates for enhanced strategies of Bias in AI to address the issue of fairness in the use of AI.