Mitigating Hallucination Issues in Small-Parameter LLMs through Inter-Layer Contrastive Decoding

In this paper, we introduce a new decoding method to mitigate the issue of hallucinations in Large Language Models (LLMs). Specifically, our method dynamically selects appropriate internal layers(DiLa) of the model and compares their logic to enhance the authenticity and accuracy during the decoding process. Experimental results show that DiLa significantly improves performance across multiple tasks without fine-tuning the model and without substantially reducing decoding efficiency. Overall, our approach is a simple yet efficient solution to alleviate the hallucination problem in LLMs.