Most current Wyner-Ziv (WZ) video codecs are based on channel coding and work in a frame by frame manner. That is, bit planes from pixels in a whole frame or coefficient band are channel coded. This structure of WZ video coding has the drawback that it can not adapt well to the spatially variable characteristics of a video frame, and hence leads to performance loss. To overcome the drawback, we propose, in this paper, a block based WZ video codec. The proposed codec works on a block by block basis, which has the advantage of better performance over the existing frame based ones. In addition, the proposed codec is also much faster than existing frame based WZ video codecs. We also demonstrate in the paper a rate distortion based encoder mode decision method in the proposed WZ video codec.