The DEEP-ER storage system uses Fraunhofer’s Parallel File System BeeGFS (formerly known as FhGFS) with its distributed metadata architecture as a basis for parallel I/O.
To take performance and scalability to the next level, the project sets two major goals for the file system layer: avoid as much communication as possible with the centralized parallel storage servers and integrate fast new limited-capacity storage technologies like NVM. To achieve these goals, the file system is split into two tiers, where the upper tier acts as a partitioned non-coherent caching layer based on NVM devices to provide very high throughput and linear scalability, while the lower tier provides the high capacity for longer term data storage based on traditional hard drives. For optimal usage of the cache layer, applications have the ability to control the staging of data between the tiers.
As of December 2016, the BeeGFS Cache API for DEEP-ER is available:
- The documentation can be found on the wiki page http://www.beegfs.com/wiki/APIsOverview.
- The packages can be downloaded via the BeeGFS website http://www.beegfs.com/release/DEEP-ER/.