PI: Matt Bement
Computational Sciences, Oak Ridge National Laboratory
In 2016, the Department of Energy’s Exascale Computing Project, or ECP, set out to develop advanced software for the arrival of exascale-class supercomputers capable of a quintillion (1018) or more calculations per second. That meant rethinking, reinventing and optimizing dozens of scientific applications and software tools to leverage exascale’s thousandfold increase in computing power. That time has arrived as the first DOE exascale computer — the Oak Ridge Leadership Computing Facility’s Frontier — opens to users around the world. “Exascale’s New Frontier” explores the applications and software technology for driving scientific discoveries in the exascale era.
The Scientific Challenge
Additive manufacturing, also known as 3D printing, offers an attractive means of building intricate machinery such as airplane landing gear or rocket parts at potentially unprecedented speed and scale. Making the most of this manufacturing method requires the ability to certify the performance and reliability of such parts. That certification can be a time-consuming and costly process. High-resolution digital simulations can help predict performance, but modeling the layer-by-layer printing and solidification of metal during 3D printing calls for more detailed simulations and more complicated calculations than available via industry software tools.
Why Exascale?
The ExaAM project is a joint effort of scientists at Oak Ridge, Lawrence Livermore and Los Alamos national laboratories and the National Institute of Standards and Technology. This project seeks to fill the gap with solutions achieved via the power of exascale computing on the Frontier supercomputer at DOE’s Oak Ridge National Laboratory. The ExaAM team has spent years honing a suite of codes and software tools to fully leverage Frontier’s world-leading speeds.
ORNL’s Matt Bement, left, and Alex Plotkowski of the ExaAM team.
“Materials qualification for nuclear plant equipment or an aircraft part by traditional methods can take years or even decades of testing,” said Matt Bement, an ORNL computational scientist who led the ExaAM project. “Additive manufacturing offers the ability to more easily manufacture components with complicated geometries that we couldn’t make at scale before. But because the 3D printing process is complicated and has a lot of parameters the operator can set (like how much power goes to the laser and how fast it moves), we need these computational tools so we can have confidence the final parts will perform as expected.”
To achieve that kind of confidence, the team needed a computer that could map the microstructures of 3D printed parts down to the nanometer (a millionth of a millimeter) from start to finish in time steps measured by the nanosecond (a millionth of a millisecond) over hours of production.
“Exascale computing gives us the means to simulate that level of detail,” Bement said. “Our ability to develop a reliable model with reproducible results helps us do a much better job of guaranteeing we can actually make the part we think we are making and that the product will perform how we expect it will. We couldn’t have done this otherwise.”
Frontier Success
The ExaAM codes have run calculations across 8,000 of Frontier’s more than 9,000 compute nodes. The codes have modeled a range of scenarios for what conditions these parts might encounter, which conditions might lead to a failure under stress and what improvements might make the parts last longer. The computational power of Frontier allowed researchers to pack what would have been months or even years of work on a typical workstation into just hours.
“The focus of our study on Frontier was understanding how 3D printing process parameters affected the metal microstructure and then understanding how the microstructure affected the strength, so there were many specimens but all with the same geometry,” Bement said. “We were able to simulate detailed models under an enormous number of conditions. That allowed us to quantify uncertainty across a wide range of parameters with a faster time to solution than would have ever been possible before. Understanding this uncertainty is absolutely essential to being able to qualify 3D printed parts for critical applications such as aircraft and nuclear reactors.”
What’s Next?
The ExaAM team has tested its predictions on a test part devised by NIST.
“The results we modeled for the benchmark part were very similar to what NIST actually observed experimentally,” Bement said. “We’ll use the data from these runs to inform the modeling and sharpen our predictions even further.”
The team has made its codes available online, and industry partners such as General Motors have adopted some codes for use on production lines.
“The ultimate goal is to make this kind of precision manufacturing an everyday phenomenon — for better processes and better products,” Bement said.
Support for this research came from the ECP, which is a collaborative effort of the DOE Office of Science and the National Nuclear Security Administration, and from the DOE Office of Science’s Advanced Scientific Computing Research program. The OLCF is an Office of Science user facility at ORNL.
UT-Battelle LLC manages ORNL for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. The Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit https://energy.gov/science.
