DAPC: A Prefix-Based Approach with Reinforced Attention for Attribute-Based Controllable Text Generation

Prefix-tuning with large language models has shown promise for enhancing attribute-based Controlled Text Generation (CTG), which aims to generate text with specific attributes. While previous works have achieved impressive results in language generation, they often suffer from performance decay across different generation length settings. In this work, we propose DAPC, a novel, lightweight prefix-based framework for CTG. The core idea of DAPC is to strategically enhance the controllability of prefixes, thereby improving the attribute distributions in the generated text. Specifically, we employ prefix-tuning through reinforcement learning to generate attribute-specific prefixes, ensuring more precise control over the target attributes. These single-attribute prefixes can easily be extended to multi-attribute prefixes for multi-attribute CTG through concatenation. Furthermore, we adopt an actor-critic framework to train a control module that dynamically adjusts the attention weights of prefixes. DAPC significantly reduces computational and storage overhead compared to previous complex methods. Experiments across various CTG tasks demonstrate that DAPC achieves significant and consistent improvements, with accuracy gains of $1 \%-15 \%$, while maintaining high fluency in tasks with different text generation length.