The amount of online video is increasing tremendously nowadays. For the convenience of information retrieval, video similarity search has become an important research issue in content-based video retrieval. There is still no satisfying scalable fast similarity search method for large database. In order to solve two challenging problems: similarity measure and fast search, a novel efficient video similarity search algorithm is proposed in this paper. A compact video image signature was computed according to the statistics of spatial-temporal distribution of video frame sequences. The video similarity is measured based on the calculation of the number of similar video components. For the scalable computing requirement, a novel efficient search method based on clustering index table was presented by index clustering. The experimental results from the query tests in large database show this method is highly efficient and effective for similar video search.