In this paper, we propose a cross-layer optimization scheme for delivery of scalable video over multirate wireless networks, in particular the popular 802.11 based wireless local area network (WLAN). The 802.11 based networks use a link adaptation mechanism in the physical layer (PHY) to maintain the reliability of transmission under varying channel conditions. When channel condition worsens, the reliability is maintained by employing more robust modulation and coding schemes, at the cost of reduced PHY bit rate. The reduced bit rate will result in lower available throughput for applications. For scalable video streaming applications, the conventional solution to this problem is to reduce the video bit rate by dropping the higher enhancement layers of the scalable video. We show in this article that the video quality can be improved, if the link adaptation scheme uses more intelligent reliability criteria and adjusts the PHY parameters used for delivering each video layer, according to the relative importance of that layer. Our scheme achieves better video quality without increasing the traffic load of the WLAN. For this purpose we present temporal fairness constraints and formulate an optimization problem for assigning different PHY modes to different layers of scalable video; the solution to this problem provides a set of PHY configuration parameters that achieve the highest possible video quality while meeting the admission control constraints in the network. Performance evaluations demonstrate that our method outperforms the existing mechanisms.