Many group communications require a security infrastructure that ensures multiple levels of access privilege for group members. Access control in hierarchy is prevalent in multimedia applications, which consist of users that subscribe to different quality levels or different sets of data streams. We present a multi-group key management scheme that achieves such a hierarchical access control by employing an integrated key graph and by managing group keys for all users with various access privileges. Compared with applying existing tree-based group key management schemes directly to the hierarchical access control problem, the proposed scheme significantly reduces the communication, computation and storage overhead associated with key management and achieves better scalability when the number of access levels increases. In addition, the proposed key graph is suitable for both centralized and contributory environments.