Paired-Prompts: Guiding LLM to Make Better Dialogue Summarization in Parameter-Efficient Fine-Tuning

Parameter-Efficient Fine-Tuning (PEFT) adapts large language models (LLMs) to downstream tasks by updating only a small number of additional parameters. However, in dia-logue summarization, the scattered key information leads PEFT methods to overfit superficial patterns, generating summaries that sacrifice informativeness for brevity. We propose Paired-Prompts, using strict training prompt to enforce conciseness, and simplified inference prompt to recover omitted details. Experiments on DialogSum and SAMSum show our method achieves significant improvements in ROUGE scores over standard PEFT, while maintaining competitive BLEU scores. This demonstrates that our method effectively balances conciseness and informativeness in dialogue summarization by using different prompts across phases.