Reliability Issues of LLMs: ChatGPT a Case Study

ChatGPT is a groundbreaking artificial intelligence (AI) invention, and this technology will see tremendous growth per the IEEE Computer Society’s 2024 technology predictions report.1 According to the report, generative AI applications top the list, and this paradigm is predicted to experience most of the advancements in the coming years. ChatGPT, a generative AI product, has demonstrated its effectiveness in many ways (e.g., answering questions, summarizing text, generating computer code, fixing programming bugs, and generating synthetic data). Despite the many promising applications, ChatGPT cannot produce desirable results for many difficult and pragmatic tasks [1]. For example, the inaccuracy from ChatGPT answers related to the emotional text is significantly high, owing to limited amounts of data—or no available data—concerning these tasks [1]. Similarly, ChatGPT can be manipulated to generate fake content, which can be hard to distinguish from real content. There are two schools of thought in the AI community about ChatGPT technology.