Symmetry is useful as a constraint in designing complex systems such as distributed controllers for multilegged robots. However, it is often difficult to determine which symmetries are appropriate. It is therefore desirable to design such systems automatically, e.g., by utilizing evolutionary algorithms that produce symmetry through developmental mechanisms. The success of these algorithms depends on how well they explore the space of valid symmetries. This paper presents an approach called evolution of network symmetry and modularity (ENSO) that utilizes group theory to search the space of symmetries effectively. This approach was evaluated by evolving neural network controllers for a quadruped robot in physically realistic simulations. On flat ground, the resulting controllers are as fast as those having hand-designed symmetry, and significantly faster than those without symmetry. On inclined ground, where the appropriate symmetries are difficult to determine manually, ENSO produced significantly faster gaits that also generalize better than those of other approaches. On robots with a more complicated structure including knee joints, ENSO resulted in more regular gaits than the other approaches. These results suggest that ENSO is a promising approach for evolving complex systems with modularity and symmetry.