Researchers have put more emphasis on stream-scheduling policies to build high performance video servers. But the most important aim of these traditional stream-scheduling schemes is to improve the resource utilization, such as memory, network bandwidth and CPU, to maximize the number of concurrent streams. These schemes did not consider the QoS needs of video streaming or described the QoS parameter weakly, so the average qualities of media streams have no guarantees when the number of media streams is very large. In this paper, we propose a new two-layer stream-scheduling scheme with QoS control for cluster video servers. We first propose new definitions for video streams with the consideration of QoS and then describe the scheme of how to schedule video streams from two layers. The target of this scheme is to improve the average QoS of all streaming tasks and to obtain large number of scheduled video streams.