This paper presents a novel method for protecting H.264/AVC streaming. The issues on the compression domain video encryption have been pointed out and fully addressed. To optimize the tradeoff between security level and computational complexity, only the most significant bits for reconstructed video quality in H.264/AVC streaming are extracted and encrypted, including the codewords of intra-prediction mode, the low frequency DCT coefficient of intra-frames and motion vector difference (MVD). To keep the encrypted streaming format-compliance to the H.264/AVC standard decoder, the codewords of intra-prediction mode are encrypted with IPME, the sign bit of the codewords of DCT coefficient are encrypted and the info_suffix of the codewords of MVD are encrypted. Experimental results show that the proposed scheme exhibits significant computational efficiency and reliable security, can resist not only perceptual attacks but also brute-force attacks. Thus, the proposed scheme will be well suited for real-time and resource-limited systems.