Covariances computation in the Gaia AVU–GSR Parallel Solver with I/O techniques: a performance study as a function of writing cycle length

The solver module of the Astrometric Verification Unit-Global Sphere Reconstruction (AVU-GSR) pipeline aims to find the astrometric parameters of $\sim 10^{8}$ stars in the Milky Way, the attitude and instrumental settings of the Gaia satellite, and the parametrized post Newtonian parameter $\gamma$ with a resolution of 1 0 - 100 micro – arc seconds. To perform this task, the code, which runs in production on Leonardo CINECA infrastructure, solves a system of linear equations with the iterative LSQR algorithm, where the coefficient matrix is large (10-50 TB) and sparse and the iterations stop when least square convergence is reached. The solver was ported to GPU with CUDA, obtaining a $\sim 14 x$ acceleration factor over an original version CPU-parallelized with OpenMP. This work concentrates on a code section dedicated to covariances calculation, representing an important scientific task for Gaia mission, since the problems unknowns present strong correlations. Given the number of unknowns at mission end, the variances-covariances matrix is expected to occupy $\sim 1$ EB, which represents a substantial “Big Data” issue. To compute a subset of the total covariances, we defined an I/Obased pipeline made of two jobs. The first job, the LSQR, writes the files every $i \operatorname{tnCov} C P$ iterations, and the second job reads them and calculates the corresponding covariances. The two jobs can be launched either in sequence or concurrently. Previous studies demonstrated that the covariances calculation does not significantly slowdown the AVU-GSR production up to $\sim 3 \times 10^{7}$ covariances. Here we investigate the performance of the covariances pipeline as a function of $i t n \operatorname{Cov} C P$. The results show that writing smaller files more frequently or writing larger files less frequently does not affect the global performance of the solver, whose speed only depends on the number of covariances to calculate and of system unknowns.