Reliability and Suitability of Evaluation of Speech by LLM Using Voice Mode

LLMs (Large-scale Language Models), such as ChatGPT, have been widely used for a variety of purposes. The current sophisticated LLMs have a voice mode, which allows a user to input and output messages using speech. In this paper, we focus on ChatGPT-4, which is one of the most popular LLMs, and investigate the suitability of the responses obtained by the LLM with its voice mode. In particular, we evaluate and explore the evaluation results of the input speech data. Through our evaluation, we show that the LLM's responses are very human-like and too gentle to evaluate the speech audio data.