Predicting Future Accesses in XRootD Caching Systems using ML-Based Network Pattern Analysis

The data generated by HEP experiments has grown exponentially in recent years, driven by new discoveries and continuous advancements in infrastructure. Intelligent techniques in data movement are required to not only improve the transfer latencies for the data but also for proper network bandwidth utilization. The current in-network caching deployments address this issue by placing the data near the computation via strategies like redundancy and strategic placement of the caches. However, the current state-of-the-art in caching strategies are rudimentary and lack the required intelligence. In this work, we leverage the rich information provided by the transfer records collected from the HEP caching framework. We design data preprocessing techniques to extract relevant information like data transfer patterns and feed them to our Machine Learning (ML) model. Our ML model is an LSTM- and GRU-based Ensemble network, specifically designed to predict the most likely datasets that will be accessed in the near future. We present the results of our approach over approx. 35 million real-world cache transfer records were collected from the HEP experiments on the Open Science Grid (OSG) over a span of one week. Our approach allows for the design of effective caching strategies like intelligent data placement, prefetching, and adaptive eviction strategies.