The Scalable Video Coding (SVC) provides the spatial, temporal, and quality scalability of SVC with which a terminal station can have different playback video quality dynamically according to the radio channel condition and the capacity of the terminal station. Under the concept of layer-separation of the OSI layer model, however, existing access network techniques are upper layer-unaware. While the traffic loading in the access network becomes heavy, the video quality degrades dramatically. In this paper, a cross multi-layer mechanism is proposed to let the MAC layer of an IEEE 802.16e network system become aware to the importance of decoding dependency of each Network Abstraction Layer unit. Besides, according to the feature of SVC, the concept of seamless service is mentioned in this paper as well. According to the simulation results, the playback video quality is improved significantly.