An optimal frequency domain bit allocation technique is presented. A Lagrange multiplier based approach is used for the rate-distortion optimisation of a quantisation matrix in video coding. Since the optimal quantisation matrix is heavily dependent on the choice of quantiser, the quantiser is also jointly optimised in order to alleviate the interaction between the two. The complexity of the algorithm is analysed, and it is shown to be directly proportional to the number of possible quantisation matrices and quantisation scales. Experimental results comparing the new technique with a conventional video encoder are also presented.