FeaMix: Feature Mix With Memory Batch Based on Self-Consistency Learning for Code Generation and Code Translation

Data augmentation algorithms, such as back translation, have shown to be effective in various deep-learning tasks. Despite their remarkable success, there has been a hurdle to applying data augmentation algorithms to code-related tasks since code consists of discrete tokens with uniqueness and certainty. In this work, we propose FeaMix, a novel yet simple data augmentation approach designed for the feature mix with memory batch based on self-consistency learning. FeaMix has a couple of uniqueness. First, it specially selects the samples to be mixed by memory batch to guarantee that the generated features are in the same spatial distribution as the mixed features. Second, it extends the self-consistency learning technique to optimize the language model for code-related tasks. With extensive experiments, we empirically validate that our method outperforms several baseline models and traditional data augmentation methods on code generation and code translation. It is noteworthy that we achieve state-of-the-art results in the CoNaLa and CodeTrans benchmarks, with a significant improvement of 1.9% in the Exact Match accuracy metric for code translation tasks.