Beyond ROUGE for Legal-Domain-Specific Evaluation Metrics for Judicial Text Summarization

Legal text summarization demands factual accuracy, statutory relevance, and jurisprudential alignment—dimensions not fully captured by standard metrics like ROUGE or BERTScore. This paper proposes a suite of legal-domain-specific evaluation metrics, including Citation Density, Statutory Inclusion Rate, Precedent Alignment Score, Factual Consistency Index, and Legal Entity Preservation Rate. We benchmark zero-shot, fine-tuned, and retrieval-augmented generation models on Indian Supreme Court judgments. Results show that domain-specific retrieval and legal entity guidance significantly improve summarization quality. We advocate for broader community adoption of legal-domain-aware evaluation to advance trustworthy and professionally usable legal AI systems.