An ORNL-led team recently received an R&D 100 Award for their work on ExaDigiT, a first-of-its-kind integrated framework that allows up-to-the-second virtual representation of computing centers for maximum performance.
A team led by researchers from the Oak Ridge Leadership Computing Facility at the Department of Energy’s Oak Ridge National Laboratory recently received an R&D 100 Award for their work on ExaDigiT, a first-of-its-kind integrated framework that allows up-to-the-second virtual representation of computing centers for maximum performance.
The software, developed in partnership with Hewlett Packard Enterprise, delivers the first complete digital twin framework for modeling data centers of supercomputers like ORNL’s exascale system Frontier in near-real time.
Modeling systems prior to ExaDigiT offered small-scale, narrower models restricted to single perspectives such as power only or cooling only. ExaDigiT combines modeling of workloads, power and cooling with visualizations and an interactive dashboard to allow optimizations, what-if analysis and virtual prototyping that can be tailored to an individual system’s needs.
“It’s basically a cluster of models that work with one another to offer a single solution that solves many different problems,” said Wes Brewer, a senior researcher for ORNL’s National Center for Computational Sciences, which oversees the OLCF. “For example, we keep track of every chip – every CPU, every GPU and the power each uses – along with the entire cooling system. These models communicate, and this constant flow back and forth allows us to predict cooling rates and tailor that to power consumption. We’re also working on the ability to model network congestion so we can schedule simulation runs on Frontier to avoid data bottlenecks and make the best use of the machine.”
Members of the ExaDigiT team, left to right: Jesse Hines, Matthias Maiterth, Wes Brewer, Feiyi Wang, Vineet Kumar, John Holmen, Rafal Wojda, Scott Greenwood, Woong Shin. Credit: Cindi Zdrinc/Oak Ridge National Laboratory
The system has been employed not only by Frontier but by other leadership-class supercomputers worldwide. HPE has used ExaDigiT to create digital twins for data centers that include Finland’s Large Unified Modern Infrastructure, or LUMI system; France’s Adastra system; and Australia’s Setonix system, among others.
Other researchers on the project team included Matthias Maiterth, Vineet Kumar, Sedrick Bouknight, Rafal Wojda, Scott Greenwood, John Holman, Jesse Hines, Woong Shin, David Grant, Wesley Williams and Feiyi Wang of ORNL and Cullen Bash, Avisek Naug, Soumyendu Sarkar, Tim Dykes, Bob Lantz, Puneet Sharma and Dejan Milojicic of HPE.
Support for this research came from the DOE Office of Science Advanced Scientific Computing Research program. The OLCF is a DOE Office of Science user facility.
UT-Battelle manages ORNL for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. The Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit energy.gov/science.
