Highly-scalable, parallel I/O will be an important building block for future Exascale systems. It will be crucial to reduce the time spent in I/O operations to a minimum and thus give way for actual compute processes. In the DEEP-ER project, we propose an I/O subsystem that relies on 3 closely integrated components to achieve exactly this: Fraunhofer's parallel file system BeeGFS, the parallel I/O library SIONlib, and Exascale10 for collective I/O operations.
Whereas the structure of the I/O system and its integration into the overall DEEP-ER architecture design allow for quite some flexibility (see graphic below), the most common scenario will be for applications to use SIONlib for task-local I/O or E10 for collective MPI-I/O based on e.g. parallel HDF5. BeeGFS then acts as the compound layer between SIONlib or E10 and the hardware architecture to ensure seamless and effective communication between the applications and the underlying hardware, e.g. NVMe.
The combined use of SIONlib and E10 will considerably improve the I/O performance compared to today’s I/O systems. Merged with the additional functionalities of BeeGFS this will boost performance even more and above all make it more scalable.
A very close co-design approach between the expert teams for BeeGFS, SIONlib and E10 ensures that the complex design of the I/O stack does not only provide for high performance and scalability but also for great usability on the side of the application developers.
Find more detailed information on the single components below.