Understanding How ChatGPT Memorizes Chinese Tang Poetry

The recent rapid development of large language models (LLMs) has shown remarkable and fascinating novel features owned by novel LLMs like ChatG PT. In this paper, the author aims to explore the memorization behavior of novel LLMs like ChatGPT. A special data type (i.e. Chinese classical Tang poetry) is used to build the test set to perform the memory extraction on ChatG PT. I also explore how the prompts during the test can affect the memory extraction results. The test indicates that the memory behavior of Tang poetry of ChatG PT follows a similar pattern to that of humans. Moreover, proper prompts can significantly affect the accuracy of extracted memories.