The High-Performance Storage System (HPSS)
Categories: Data Management, Data Transfer, File Systems
Print this article
The High Performance Storage System (HPSS) at the OLCF provides longer-term storage for the large amounts of data created on the OLCF compute systems. The mass storage facility consists of tape and disk storage components, servers, and the HPSS software. All data persists on disk for a period after it is uploaded that is determined by how full the disk caches become. When data is migrated to tape it is done so in a first in first out fashion.
Each OLCF user receives an HPSS account automatically. Users can transfer data to HPSS from any OLCF system using the HSI or HTAR utilities. Initially, data transferred to HPSS is written to disk; the system then migrates the data to tape for longer-term archival.
HPSS hasSL8500 tape libraries, each holding up to 10,000 cartridges. The libraries house a total of (24) T10K-A tape drives (500 GB cartridges, uncompressed) and (60) T10K-B tape drives (1 TB cartridges, uncompressed), (36) T10K-C, and (72) T10K-D. Each drive has a bandwidth of 250 MB/s.
ORNL’s work in mass storage began in the early 1990s to support the Atmospheric Radiation Measurement project and to provide storage for simulation results generated on the NCCS’s Paragon supercomputers. To support those projects, ORNL acquired and ran the NSL UniTree storage management product.
In 1993 a follow-on to NSL UniTree, known as HPSS, was being designed by IBM and a collaboration of Department of Energy (DOE) national laboratories (Sandia, Livermore, and Los Alamos). ORNL joined that collaboration and took on responsibility for the storage system management (SSM) portion of the product, for which the ORNL HPSS development team continues to be responsible.
ORNL continued with NSL UniTree production use until 1997, at which time the conversion to HPSS was completed. In 1997, HPSS won an R&D 100 Award based on an entry initiated and prepared at ORNL.
As storage, network, and computing technologies continue to change the OLCF’s storage system evolves to take advantage of new equipment that is both more capable and more cost-effective.