DEEP stands for Dynamical Exascale Entry Platform. Since 2011, the DEEP series of projects (DEEP, DEEP-ER, DEEP-EST) has pioneered an innovative concept known as the Modular Supercomputer Architecture (MSA). This novel HPC architecture paves the way for the next milestone in supercomputing: exascale systems with the power to perform 10^18 Floating Point Operations per second.
DEEP-SEA: Software for Exascale Architectures
The fourth project in the row is DEEP-SEA. It started on April 1, 2021 with a term of 3 years. The project will deliver the programming environment for future European exascale systems, adapting all levels of the software stack to support highly heterogeneous compute and memory configurations. It will also allow code optimisation across existing and future architectures and systems. The software stack includes low-level drivers, computation and communication libraries, resource management, and programming abstractions with associated runtime systems and tools.
With DEEP-EST, a prototype was established that leverages the benefits of a Modular Supercomputing Architecture (MSA). Here, different components like standard CPUs, accelerators like GPUs, among others, build a complex mesh of heterogeneous technologies. Each technology is best suited for specific workloads, applications can leverage the parts of the MSA that allow for optimal energy/performance ratio. Large-scale simulations, data analytics, machine- and deep learning – different jobs have different requirements to run at the equilibrium of energy usage and compute performance.
The downside: the complexity on system and node level makes it hard to allocate all resources in the best way. Parallelization is not trivial in traditional HPC environment – with the rising complexity of heterogeneous resources is will be hardly manageable by the users and the application developers. They need to understand their codes and the underlying hardware very well in order to decide on what components parts of the code should run. Also, the need grows to port codes to different technologies such as accelerators or even to completely new platforms. Not to mention the burden of optimization for a variety of HPC systems. DEEP-SEA addresses this challenge.
The main goals of this project are:
- Better manage and program compute and memory heterogeneity.
- Targets easier programming for Modular Supercomputers.
To achieve these goals, DEEP-SEA looks at all relevant parts of the system: nodes, clusters, applications, system software and tools. To provide solutions that are usable on as many HPC systems as possible and that scale to exascale and beyond, DEEP-SEA uses the co-design approach that has already proven effective in earlier DEEP projects.