Rapid Object Modeling Initialization for Vector Quantized-Variational AutoEncoder

This paper presents a novel approach for codebook initialization in the well-established Vector Quantized-Variational AutoEncoder (VQ-VAE). Rapid Object Modeling (ROM), inspired by few-shot metric learning, leverages the knowledge base of pretrained deep neural networks to rapidly generate object appearance models that are aggregated into reference concept dictionaries. The results of this paper explore the effects of codebook size and embedding vector length on reconstruction error, followed by a qualitative comparison between the reconstructed images. Initializing the VQ-VAE with ROM generated concept dictionaries showed several desirable improvements, including 47% lower reconstruction errors, faster convergence, and more predictable performance.