Convergence Analysis of Dirac-GAN

Generative adversarial networks are powerful and widely used generative models, but they often face challenges in achieving convergence during the training process, which may lead to failures. As a univariate model of GANs, Dirac-GAN can effectively capture the relevant properties of GANs and significantly alleviate the analysis complexities. In this paper, we analyze the convergence of Dirac-GAN with regularization terms and gradient descent training. We leverage control theory and local linearization methods to establish conditions for the convergence of Dirac-GAN. These conditions constrain the value range of two crucial parameters, namely the regularization coefficient and learning rate, ultimately allowing us to derive the convergence rate of Dirac-GAN. We provide specific analyses of the objective functions of five common GANs, determining the corresponding convergence conditions and the optimal learning rate for achieving the fastest convergence. Finally, the theoretical results are validated through experiments.