It has been shown that incorporation of human-specified high-level description of the target objects, e.g. labeled prior-knowledge data, can increase the performance of one-shot recognition. In this paper, we introduce latent components as a high level representation of the original objects and propose a cascade model for one-shot image recognition based on latent components learned by Hierarchical Dirichlet Process (HDP). In the proposed approach, instead of solving an optimization problem in the training stage, the latent high-level components are learned efficiently in a unsupervised way from unlabeled prior-knowledge data. Motivated by the facts that HDP is an infinite mixture model proposed in the literature for document modeling that can infer the unknown mixture components and the number of components from the data, and that bag-of-feature model is a standard representation in document retrieval and computer vision areas, we adopt HDP model to infer the mixture components (like latent topics in documents) for target images from unlabeled image visual word vocabulary, and we then train a classifier to associate the components with class labels. The superior performances of the proposed one-shot recognition method are illustrated by testing the Caltech category dataset and the " Animals with Attributes" dataset.