Framework for High-Performance Data Transfers Optimization in Large Distributed Systems

Handling high-performance data transfers is a hot topic today, in the context of the numerous data-intensive applications which begin to be used not only in local clusters, but also in large distributed systems. For this purpose, we have developed a framework based on MONALISA that allows intelligent handling of data transfers in large environments, both over current and next-generation, hybrid high-speed networks. This paper presents its design and two policies for handling the data transfers contention to network resources. We argue for its effectiveness by presenting our experiences with the framework, using the FDT data transfer application, in the context of the ALICE Grid and the US LHCNet network.