Share File Updating for Test Farms Using a File Cache

When a file needs to be updated by thousands of tests running on a compute farm, waiting for a chance to update slows down the overall runtime as the number of parallel test runs increases. We implemented a file cache that allows each test to write a small temporary file that either updates the shared file or is stored for later updating. A network file lock controls access to the shared file. If the file lock is not acquired, temporary files are merged to reduce the number of future file reads. Temporary files are removed only if their content is successfully added to the shared file or merged into another temporary file. Each test needs to update the shared file many times during a run, so a version number is used to select the temporary file to read and discard the others. The file cache eliminated the time delay for updating the shared file, saving hours of runtime per test suite and speeding up developer check-in time with no penalty for increasing tests or compute nodes. This paper will describe how the file cache was developed, detail the current approach, and illustrate the performance benefits.