Legal Document-Based, Domain-Driven Q&A System: LLMs in Perspective

Question Answering systems based on large language models are widely employed today, benefiting from continuous enhancements and improved performance. The legal domain has become a particularly active focus for Question Answering systems, given its complexity and social importance. This paper offers a discussion on how larger and smaller language models can be used to build a legal document-based Question Answering system. We present a novel model, named Cocoruta, generated by fine-tuning with a corpus of legal documents. In addition, we examine five LLMs as they answer questions related to the legal aspects of a specific domain – the Blue Amazon, a region of particular interest involving environmental issues. The results suggest that while LLMs are not yet of sufficient quality for use as core in legal context Question Answering systems, fine-tuning on specialized corpora imparts a beneficial bias to their legal discourse. Despite having fewer parameters, the Cocoruta model competes well with larger LLMs in this aspect.