Recently, tracking is regarded as a binary classification problem by discriminative tracking methods. However, such binary classification may not fully handle the outliers, which may cause drifting. In this paper, we argue that tracking may be regarded as one-class problem, which avoids gathering limited negative samples for background description. Inspired by the fact the positive feature space generated by One-Class SVM is bounded by a closed sphere, we propose a novel tracking method utilizing One-Class SVMs that adopt HOG and 2bit-BP as features, called One-Class SVM Tracker (OCST). Simultaneously an efficient initialization and online updating scheme is also proposed. Extensive experimental results prove that OCST outperforms some state-of-the-art discriminative tracking methods on providing accurate tracking and alleviating serious drifting.