Can We Build a Generative Model without Back Propagation Training?

Driven by deep learning, the field of content generation has witnessed remarkable progress. However, it still faces several challenges, such as low efficiency and high difficulty in training. To confront these obstacles, we propose an efficient and effective lightweight generative model, termed the pseudoinverse learning based variational autoencoder, within the framework of the synergetic learning system. The proposed learning system in this study comprises a reductive subsystem and a generative subsystem, incorporates a non-gradient learning scheme. The reductive subsystem employs variants of the pseudoinverse learning algorithm and probabilistic principal component analysis to embed the inputs into the latent space which is constrained to follow a standard normal distribution. The generative subsystem performs the inverse reconstruction process of the reductive subsystem, which can be used for content generation after training. The experimental results show that the proposed model significantly speeds up the training while achieving comparable generation quality to the baselines.