PI: Danny Perez,
Los Alamos National Laboratory
In 2016, the Department of Energy’s Exascale Computing Project (ECP) set out to develop advanced software for the arrival of exascale-class supercomputers capable of a quintillion (1018) or more calculations per second. That leap meant rethinking, reinventing and optimizing dozens of scientific applications and software tools to leverage exascale’s thousandfold increase in computing power. That time has arrived as the first DOE exascale computer — the Oak Ridge Leadership Computing Facility’s Frontier — opened to users around the world. “Exascale’s New Frontier” explores the applications and software technology for driving scientific discoveries in the exascale era.
The Science Challenge
The design of both fission and fusion reactors raises important questions for engineers: Which materials best withstand the extreme conditions that result from fission and fusion interactions? And how long can these materials be subjected to these conditions and still be effective?
Although fission reactors have been contributing to the nation’s power grid since the 1950s, they still only burn around 5% of the uranium atoms in their primary fuel, uranium oxide. This is partly due to the pellet-cladding interaction, in which fuel pellets swell from radiation damage, which ultimately causes their cladding to fail. A better understanding of how this nuclear fuel evolves could lead to designs with stronger materials, thereby enabling much higher burnup levels and reduced nuclear waste.
Meanwhile, proposed fusion reactors promise many advantages over their fission counterparts, including plentiful fuels, high reliability and efficiency, and significantly fewer risks (e.g., no meltdowns). However, to replicate the process that powers the stars, fusion reactors will require advanced structural materials that can resist very high temperatures and extremely hot plasmas.
Why Exascale?
The EXAALT (Exascale Atomistics for Accuracy, Length, and Time) effort aims to help researchers find the best materials for constructing fission and fusion reactors. Through molecular dynamics (MD) simulations, EXAALT will enable scientists to explore at the atomic scale how the structures of different materials will evolve in the harsh conditions typical of these reactors.
EXAALT combines three open-source codes to form a state-of-the-art MD simulation software stack: LAMMPS (Large-Scale Atomic/Molecular Massively Parallel Simulator) is the main computational engine, LATTE (Los Alamos Transferable Tight-binding for Energetics) is used for quantum-level simulations, and ParSplice is the management layer that orchestrates the simulations. Tracking every atom in a system over long timescales and at quantum levels of accuracy requires incredible supercomputing power.
“There’s a sweet spot where we have accurate-enough models, long-enough timescales, and large-enough system sizes that really only become accessible at exascale,” said Danny Perez, principal investigator of the EXAALT project and a technical staff member at Los Alamos National Laboratory. “If your system is too small, then you’re not going to learn what you must learn. If you don’t run long enough, then you’re not going to see the basic unit steps that let your material evolve. And if you’re not accurate enough, then you really cannot say anything about what a real material will look like. So, you really need the intersection of these three ingredients, and that only became possible with exascale machines.”
This simulation shows the growth of helium bubbles (shown in purple) in tungsten in conditions relevant to fusion reactors. Achieving realistic growth rates is essential to capture the proper morphological evolution of the material’s surface. EXAALT extends simulation timescales to those that are closer to realistic conditions. Image credit: Exascale Computing Project
Frontier Success
The EXAALT project had a speedup goal of 50× compared to its pre-ECP performance in 2016. With its early runs on the Frontier exascale supercomputer this year, it produced a speedup of 400× over its baseline. Perez believes future Frontier runs, with access to more nodes, will produce an even higher figure.
What’s Next?
The EXAALT team will continue developing its code before the conclusion of the ECP. Currently, the team is optimizing the code’s GPU support to run on the upcoming Aurora exascale supercomputer at Argonne National Laboratory. Next, the team aims to further improve EXAALT’s machine learning–augmented physics models. Ultimately, Perez believes that EXAALT can make a smooth transition to industry due to the popularity of its main computational engine, LAMMPS.
“We basically built upon a set of tools that were already very broadly adopted in the community and added capability on top of it. We think that the barrier to adoption will be much lower this way because it’s an environment that people are familiar with—their existing workflows are probably already running under LAMMPS. So, I think we’ll have an easier time doing the tech transfer between the research part and the application,” Perez said.
Support for this research came from the ECP, a collaborative effort of the DOE Office of Science and the National Nuclear Security Administration, and from the DOE Office of Science’s Advanced Scientific Computing Research program. The OLCF is a DOE Office of Science user facility.
UT-Battelle LLC manages ORNL for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, visit https://energy.gov/science.
