Evaluating GPT-4’s Ability to Identify Additional Context

We evaluate the strengths and weaknesses of the state-of-the-art in LLMs when identifying additional information supplied by users in dialogue agents. While GPT-4 can accurately identify additional information in some sentences, it fails to identify additional context more than 22% of the time. By understanding these limitations, we can harness LLMs within the scope of their abilities and compare to other approaches.