A study led by scientists at the US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL) plumbs the depths of deep learning and could connect the digital synapses toward better, more reliable artificial-intelligence (AI) models.
The research team from ORNL and the National Energy Technology Laboratory used Summit, the nation’s fastest supercomputer, to train a 3D deep-learning convolutional neural network, a type of AI model that mimics the human brain by learning from data and feedback. Neural networks tend to excel at spotting patterns and at processing layers of information, such as pixels in an image or strips in a lattice. But because the networks constantly evolve as they learn, scientists too often can’t track how they reach a conclusion and can’t independently verify the results.
“Most applications treat it as a black box,” said Junqi Yin, the study’s lead author and a researcher in the Analytics and AI Methods at Scale Group at ORNL’s National Center for Computational Sciences. “The typical neural network has millions of parameters processing data in a nonlinear fashion, unlike a typical equation. It’s too intertwined to look inside and we don’t know exactly how to interpret the way the model works, so we can’t place high confidence in what it tells us. That’s a big problem. To make meaningful scientific breakthroughs, we need interpretable readouts that allow us to determine how the model reached its conclusion and to reproduce the results. We want to be able to show our work.”
The team used Summit, the Oak Ridge Leadership Computing Facility’s (OLCF’s) 200-petaflop flagship IBM AC922 supercomputer, to simulate 3D blocks of metal alloys down to the atomic level and train the model to recognize the alloys’ chemical arrangements. The team then assigned the model to design a new, high-entropy alloy that would possess the best possible mechanical properties from a vast array of molecular options.
Oak Ridge National Laboratory scientists used a deep-learning convolutional neural network trained on Summit, the world’s fastest supercomputer, to design a high-entropy alloy. The results could lead to a new generation of more reliable and transparent artificial-intelligence models for materials design. Image credit: Junqi Yin, ORNL
Simulating the alloys required the computational power of Summit to create an elaborate, multilayered system of nearly 14,000 atoms divided into eight blocks. Training on a less powerful machine would have taken about 2 days. Summit’s world-class speeds shrank that training time to 10 minutes.
Researchers could then connect details from the training simulations—such as phase transitions, or changes in the physical states of materials—to values used by the trained model to reach its conclusions. The model’s findings parallel real-world results reached in unrelated experiments by a Massachusetts Institute of Technology team that designed an alloy of titanium, vanadium, niobium and hafnium.
Yin sees the study as a step toward a new generation of more transparent and dependable AI models.
“The ultimate goal was to find new, customizable materials,” he said. “But unlike most deep-learning models, this one is interpretable because we can connect the variables in the results—what element to start with, what concentration of each element, and so on—to our physics observations used to train the model. This study shows we can build a neural network that connects directly to observable data, so we can peer inside the process, work backward to verify answers, and place much higher confidence in the results.”
The team hopes to build on their success by using the neural network as part of an autonomous workflow system that would design new materials from start to finish with little human involvement.
Support for this research came from the DOE Office of Science’s Advanced Scientific Computing Research program. The OLCF is a DOE Office of Science user facility at ORNL.
Related publication: Yin, Junqi, Zongrui Pei, and Michael C. Gao. “Neural Network-Based Parameter for Phase Transitions and Its Applications in High-Entropy Alloys.” Nature Computational Science 1 (2021): 686–93, https://doi.org/10.1038/s43588-021-00139-3.
UT-Battelle LLC manages Oak Ridge National Laboratory for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, visit https://energy.gov/science.
