A new method to extract person-independent expression feature based on higher-order singular value decomposition (HOSVD) is proposed for facial expression recognition. Based on the assumption that similar persons have similar facial expression appearance and shape, the person-similarity weighted expression feature is proposed to estimate the expression feature of test persons. As a result, the estimated expression feature can reduce the influence of individuals caused by insufficient training data, and hence become less person-dependent. The proposed method is tested on Cohn-Kanade facial expression database and Japanese female facial expression (JAFFE) database. Person-independent experimental results show the superiority of the proposed method over the existing methods.