Adaptive Sharpness-Aware Minimization for Adversarial Domain Generalization

To obtain invariant representations, domain-adversarial training has been widely employed, and it is also frequently used for a variety of domain generalization (DG) tasks. For supervised learning tasks such as classification, emerging techniques that converge to smooth optima have demonstrated increased generalization. In this study, we examine how smoothness-enhancing formulations affect DG, which aims to combine task loss with adversarial terms. We consider that stabilizing the adversarial training leads to superior performance on the target domain when converging to a smooth minima with respect to task loss. Contrary to task loss, our analysis demonstrates that smooth minima convergence with respect to adversarial loss results in suboptimal generalization on the target domain. The investigation led us to develop the adaptive sharpness-aware minimization (ASAM) for adversarial DG method, which significantly improves the performance of current domain-adversarial methods for classification tasks. The proposed strategy performs similarly to the traditional state-of-the-art options in the most recent benchmark, Digits-DG, PACS, Office-Home, and DomainNet.