Project Description
Spectral is a transparently applied library for taking advantage of the Summit Burst Buffer architecture. Applications using per-process output simply write to the node-local burst buffer. Upon closing the file, spectral automatically schedules the output to drain to the parallel filesystem, this requires no intervention from the application. This drain takes advantage of new technology in Summit nodes that enable draining to the PFS to occur silently while the application is making scientific progress.
SymphonyFS is a FUSE-based client that extends local write-caching to use a node-local NVM device. SymphonyFS provides the performance benefits of writing to local NVM while giving the application the appearance of writing to the parallel file system (PFS). The FUSE daemon then drains the data to the PFS in the background. SymphonyFS is suitable for file-per-process and shared file uses.
The IBM Burst Buffer API is a C/C++ compatible library co-designed by IBM, LLNL, and ORNL for taking advantage of Summit’s Burst Buffer Architecture. The API enables the scheduling of data movement off of the Burst Buffer and into the GPFS parallel file-system. Utilizing, NVMe-Over-Fabrics, the server components of BBAPI are able to transparently drain the data off of the Burst Buffer without interfering with the running application.
The IBM Burst Buffer Shared Checkpoint File-System is an IBM, LLNL, and ORNL co-design project, seeking to close the gap for shared file checkpointing. When utilized, a new BSCFS mount point will be created on the Summit compute nodes. Applications writing shared files from several nodes will be able to use this mount point for their files and data will be written through the node-local burst buffer and asynchronously persisted to the GPFS parallel file system. Taking advantage of the BSCFS software requires small application modifications to the I/O portions of the software, however, large applications taking advantage of BSCFS should see significant improvements in their I/O performance.
NVMalloc is a library that exposes node-local NVM as byte-addressable memory. It provides a simple, familiar API that is similar to malloc() to the application and hides the complexity of using mmapped files. NVMalloc can extend the available memory to an application beyond system memory, especially for read-heavy data.
