This paper aims at providing a key agreement protocol for smart grid to cope with access control of appliances/devices located inside a Home Area Network (HAN) by a set of controllers outside the HAN. The commands/packets initiated by the controllers in crisis cases should be delivered fast and immune from any interruption. The HAN controller, which acts as a gateway, should not cause any delay by decrypting and re-encrypting the packets, nor should it has any chance to modify them. Considering the required level of security and quality of service, we design our protocol with an Elliptic Curve Cryptography (ECC) approach. We improve and implement the Password Authenticated Key Exchange (PAKE) protocol in two steps. First, we propose an auxiliary mechanism that is an ECC version of PAKE, and then extend it to a multilayer consensus model. We reduce the number of hash functions to one, and utilize a primitive password shared between an appliance and HAN controller to construct four valid individual consensus and authenticated symmetric keys between the appliance and upstream controllers by exchanging only 12 packets. Security analysis presents that our protocol is resilient to various attacks. Furthermore, performance analysis shows that the delay caused by the security process is reduced by more than one half.