The “Pioneering Frontier” series features stories profiling the many talented ORNL employees behind the construction and operation of the OLCF’s incoming exascale supercomputer, Frontier. The HPE Cray system was delivered in 2021, with full user operations expected in 2022.
With the Oak Ridge Leadership Computing Facility (OLCF) slated to launch the nation’s first exascale system later this year, having computational codes that run efficiently on the system is one of the center’s most crucial missions.
Markus Eisenbach is a senior computational scientist at the OLCF working to port the well-established Locally Self-Consistent Multiple Scattering (LSMS) materials code to the HPE Cray EX Frontier.
LSMS is a first-principles code used to calculate the properties of materials, including magnetic materials, metallic systems, and alloys. As one of the OLCF’s Center for Accelerated Application Readiness codes, LSMS can perform physics calculations for extremely large material systems—more than 100,000 atoms—as determined by the motions of electrons in a solid. The code is currently deployed on Crusher, a 1.5-cabinet test system modeled on Frontier, and LSMS will scale to the full Frontier system when it becomes fully operational later this year.
Markus Eisenbach is developing a well-established materials code, the Locally Self-Consistent Multiple Scattering (LSMS) code, to run on the HPE Cray EX Frontier as soon as it becomes available to users. Image Credit: Carlos Jones, ORNL
“With Frontier, we will be able to perform LSMS calculations of larger systems and also study new physics,” said Markus Eisenbach, senior computational scientist at the OLCF and principal investigator of LSMS. “Because we will have significantly more computational power available with Frontier, we can actually use physics models that include more correlation effects that we can’t capture as easily on current systems.”
Eisenbach and his team are looking forward to combining classical statistical mechanics—which provides the team with the behavior of materials at different temperatures—with machine learning workflows on Frontier to more rapidly calculate material behaviors.
Eisenbach began developing LSMS in 2001 after he came to the US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL) for a postdoctoral research associate position. He had just earned his PhD in theoretical physics from the University of Bristol in England. A condensed matter physicist by training, Eisenbach had always been interested in the natural sciences. After zeroing in on physics and mathematics, he decided to combine the two interests—first landing on theoretical physics and then eventually on computational physics. As for what he enjoys the most about physics and mathematics, he said it’s the magic.
“I enjoy learning about how things work, but also there is an inherent magic in rearranging numbers and figuring out how things behave,” he said. “You can manipulate formulas and understand the consequences, so there’s a bit of curiosity and playfulness in it too.”
Today, Eisenbach is making good use of that playfulness to develop the LSMS code for Frontier in collaboration with Yang Wang of the Pittsburgh Supercomputing Center and Carnegie Mellon University. Eisenbach is currently using the code on other platforms and is implementing new capabilities to ensure that it will run efficiently at scale on Frontier.
“We want to make sure that we can calculate and implement some new items to measure,” he said. “We are also looking at additional ways of incorporating electron correlation into the code and allowing for the movement of atoms. Finally, we are reoptimizing the code and reimplementing its kernels for AMD GPUs.”
The team is interested in studying two classes of systems: (1) complex, large-scale, magnetic orders of material structures and (2) defects in materials, such as those used in alloys.
“We are generally studying these disordered systems where you don’t have a perfect alignment of all the atoms,” Eisenbach said. “With LSMS, we can calculate very large cells and model alloys where atoms are randomly placed at different sites. We can move these atoms around, compare their energies, and calculate the statistical mechanics of them. We want to know at what temperature these materials start to separate into different phases or become ordered or disordered.”
Understanding the mechanical properties of materials is important because they can change dramatically depending on whether a material has a periodic, regular arrangement of atoms or randomly arranged atoms. The defects, Eisenbach said, are important for understanding the mechanics and behavior of these materials.
“How tough is steel? How can you build a magnet with better properties for improved electric engines? How do high-temperature alloys, which have potential applications in turbines and jet engines, behave? These are the questions that this kind of research can answer,” Eisenbach said.
With Frontier, the team will be able to capture larger systems and include new physics to study materials. Although calculating the interactions of many electrons always requires some approximations, the team will be able to calculate more of their intricacies with Frontier. The team can include more mathematics, relax some of their approximations, and calculate larger systems—all with greater accuracy than ever before.
The results of such simulations could have applications in high-temperature alloys, materials for fusion reactors, and spintronic devices, which use the spin of electrons and their magnetic movements for storage devices.
UT-Battelle LLC manages ORNL for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, visit https://energy.gov/science.
