As a temporal classification problem, visual-based human actions recognition is an important component for some potential applications. In this paper, we combine Fuzzy Principle Component Analysis(Fuzzy PCA) and hidden Conditional Random Fields(HCRFs) to achieve a viewpoint insensitive human action recognition. Fuzzy PCA is used to reduce the dimension of the silhouette image features to obtain the compact representation of action space. HCRFs is applied to model the human actions from different actors and different viewpoints. This method can relax the independence assumption of the generative model. Experiment results on a public dataset demonstrate the effectiveness and robustness of our method.