Management of Online Processing Farms in the ATLAS Experiment

The ATLAS experiment will use of order three thousand nodes for the online processing farms. The administration of such a large cluster is a challenge. The ability to quickly turn on/off machines, especially after a power cut, and the ability to remote monitor the hardware health whether the machine be on or off are some of the major issues. To solve these problems ATLAS has decided wherever possible to use Intelligent Platform Management Interfaces (IPMI) for its nodes. This paper will present the mechanisms which were developed to allow the distribution of management and monitoring commands to many machines. These commands were run simultaneously on the prototype farm, by taking into account the specificities of the different IPMI versions and implementations, and the network topology. Results from timing measurements for the distribution of commands to many nodes, for booting and for shutting down of the nodes will be shown with an extrapolation to the final cluster size.