Security management is one of the five management functions defined by ISO/OSI, which covers two aspects: security of management and management of security. As to management of network security, we give security management requirements for large computer networks, combined with our management experience of managing CERNET. From our experience, the widely used firewalls in the Internet are lacking in the capability to be remotely managed on a large scale, especially multi-vendor network environment. Addressing these problems, the concept of high level security policy management is proposed for large networks. To support high level policy management and collaborative management of multi-vendor firewalls, a definition for a common firewall MIB (CFWMIB) and a common format for the TRAP events record are proposed. For the aspect of security of network management, we present a security architecture which is implemented in our web-based network management system. Flexible authentication and role-based access control mechanisms of the architecture are described. Our ongoing and future research is also outlined.