On Text Granularity and Metric Frameworks for Large Language Model Content Detection

Breakthroughs in Large Language Models (LLMs) have allowed Artificial Intelligence (AI) assistant systems to provide quality information with conveniences. An issue is paralleling the advantages, however. One among the problems of LLM generated content is that they seem indistinguishable from that of human which leads to numerous issues in areas like science, education, information security, etc. Furthermore, approaches in LLM content detection are either computationally expensive or need the LLMs’ internal computations which make them more difficult to be used by the public. Addressing the research gap, we present a metric learning framework for LLM text detection that is balanced for resources, accessibility, and performance. Specifically, the detection framework relies on metric learning to evaluate the similarity between a given text to an equivalent example from LLMs and verify whether the former is from human or AI. The framework can be trained in triplets or pairs of text instances from the same contexts at either the full-text or the sentence granularity levels. For benchmarking, five corpora totalling over 95,000 contexts and responses from human and GPT-3.5 TURBO or GPT-4 TURBO are developed. In term of performance, our architectures maintain 0.87 to 0.95 F1 scores throughout multiple experiment settings. Our framework also requires much less time in training and inference compared to RoBERTa, LLaMA 3, and Ghostbuster, while having 90% to 150% performances of the best benchmark.