MAT: Medical AI-generated Text Detection Dataset from Multi-models and Multi-Methods

Large language models (LLMs) have been widely used in society due to their amazing emergent capabilities, but they also bring security issues. A large amount of content on the Internet may be generated by AI. Whether it is social forums or more professional academic fields, the abuse of AI has become a problem. Especially in some professional fields, Blindly trusting what the Internet says is dangerous. For this reason, for some fields that are risky and need to limit the use of AI, such as medicine, a more comprehensive benchmark is needed to test the ability of AI-generated text detection tasks. Considering the popularity of LLMs, the data distributions used in the training process of different LLMs may lead to the differences in generated data distributions, especially for some LLMs for non-native English speakers. To address this issue, this article introduces an AI-generated text detection dataset in the field of medical question answering. This dataset is generated by various models and prompting methods, and cross validation is performed on multiple types of data between texts generated by different methods to verify the effectiveness of AI-generated text detection and model classification tasks, and to study the generalization of the dataset in different tasks. We have published the dataset for future research on https://github.com/Hellpoop/MAT.