Enhancing Branchformer with Dynamic Branch Merging Module for Code-Switching Speech Recognition

Branchformer combines the self-attention and convolutional gating multi-layer perceptron blocks as two branches to capture global and local features. To merge both features, Branchformer applies concatenation and weighted average methods. In this work, extending from Branchformer, we propose DBM-Branchformer in a novel way for the combination of both features. DBM-Branchformer employs Dynamic Branch Merging modules (DBMs) to merge both branches. These modules fuse two branches with dynamic weights for a more flexible combination than Branchformer. On TALCS, the code-switching dataset, DBM-Branchformer obtains a word error rate of 6.9%, outperforming the Branchformer with two methods by 0.1% and 0.2%.