Layered transmission is a promising solution to video multicast over the heterogeneous Internet. However, since the number of layers is practically limited, noticeable mismatches would occur between the coarse-grained layer subscription levels and the heterogeneous and dynamic rate requirements from the receivers. In this paper, we show that such mismatch can be effectively reduced using a dynamic and fine-grained layer rate allocation on the sender's side. Specifically, we study the optimization criteria for rate allocation, and propose a metric called application-aware fairness index. This metric takes into consideration 1) the nonlinear relation between the perceived video quality and the delivered rate and 2) the degree of satisfaction for receivers with heterogeneous bandwidth requirements. We formulate the rate allocation into an optimization problem with the objective of maximizing the expected fairness index for all receivers in a multicast session. We then derive an efficient and scalable solution, and demonstrate that it can be seamlessly integrated into an end-to-end adaptation protocol, called hybrid adaptation layered multicast (HALM). This protocol takes advantage of the emerging fine-grained layered coding, and is fully compatible with the best-effort Internet infrastructure. Simulation and numerical results show that HALM noticeably improves the degree of fairness, and interacts with TCP traffic better than static allocation based protocols. More important, increasing the number of layers in HALM generally improves the degree of fairness; it is sufficient to obtain satisfactory performance with a small number of layers (three to five layers).