High bandwidth scientific data management using storage area networking: lessons learned at the Starfire Optical Range

Advances in sensor technology, electronic devices, digital processing hardware, and computation systems have vastly increased the volume of data available for collection at scientific research sites. The Starfire Optical Range of the Air Force Research Laboratory Directed Energy Directorate is responsible for the research and development of advanced techniques for atmospheric compensation for large telescopes. Recent experiments have been completed which encompassed more that 100TB of collected data. The collection, processing, archival, and analysis of these data required an entirely new approach for the facility staff. This paper presents the architecture, hardware and software components, integration difficulties, and performance results in a "lessons learned" format focused on specific implementation choices, interoperability lessons and issues, and results and capabilities.