Loading [a11y]/accessibility-menu.js
CAPIO: a Middleware for Transparent I/O Streaming in Data- Intensive Workflows | IEEE Conference Publication | IEEE Xplore

CAPIO: a Middleware for Transparent I/O Streaming in Data- Intensive Workflows


Abstract:

With the increasing amount of digital data available for analysis and simulation, the class of I/O-intensive HPC workflows is fated to quickly expand, further exacerbatin...Show More

Abstract:

With the increasing amount of digital data available for analysis and simulation, the class of I/O-intensive HPC workflows is fated to quickly expand, further exacerbating the performance gap between computing, memory, and storage technologies. This paper introduces CAPIO (Cross-Application Programmable I/O), a middleware capable of injecting I/O streaming capabilities into file-based workflows, improving the computation- I/O overlap without the need to change the application code. The contribution is twofold: 1) at design time, a new I/O coordination language allows users to annotate workflow data dependencies with synchronization semantics; 2) at run time, a user-space middleware automatically and transparently to the user turns a workflow batch execution into a streaming execution according to the semantics expressed in the configuration file. CAPIO has been tested on synthetic benchmarks simulating typical workflow I/O patterns and two real-world workflows. Experiments show that CAPIO reduces the execution time by 10% to 66% for data-intensive workflows that use the file system as a communication medium.
Date of Conference: 18-21 December 2023
Date Added to IEEE Xplore: 05 April 2024
ISBN Information:

ISSN Information:

Conference Location: Goa, India

Contact IEEE to Subscribe

References

References is not available for this document.