PI: Jacqueline Chen,
Sandia National Laboratories
In 2016, the Department of Energy’s Exascale Computing Project (ECP) set out to develop advanced software for the arrival of exascale-class supercomputers capable of a quintillion (1018) or more calculations per second. That meant rethinking, reinventing, and optimizing dozens of scientific applications and software tools to leverage exascale’s thousandfold increase in computing power. That time has arrived as the first DOE exascale computer – the Oak Ridge Leadership Computing Facility’s Frontier – opens to users around the world. “Exascale’s New Frontier” explores the applications and software technology for driving scientific discoveries in the exascale era.
The scientific challenge
Diesel and gas-turbine engines drive the world’s trains, planes, and ships, but the fossil fuels that power these engines produce much of the carbon emissions that fuel the greenhouse effect and global climate change. Scientists have spent the past half-century in search of cleaner-burning fuels but have been hindered by the complexities of the high-pressure, turbulent reacting environment inside practical combustion chambers. Electrification trends underway across the automotive industry have yet to reach harder-to-electrify energy sectors such as off-grid power generation, marine shipping, agriculture, mining, and airplanes. A key decarbonization strategy calls for replacing the petroleum-based fuels for these types of engines with fuels from sustainable sources, while maintaining rigorous requirements on reliability, safety, and cost.
Why exascale?
The Combustion-Pele project, named for the ancient Hawaiian goddess of fire, offers a means to overcome the obstacles to cleaner-burning fuels for these sectors. Over the past seven years, the Pele project team has developed simulation tools that harness the computational power of exascale to digitally recreate these complex combustion environments in unprecedented detail. The project began with a focus on finding ways to improve the internal combustion engines that power cars and trucks but pivoted to include aviation systems.
An artist’s rendering of the Frontier supercomputer. Credit: Sibling Rivalry/HPE
Current efforts focus on simulating the combustion process inside a jet engine that burns biologically derived fuels and on understanding how differences in fuel properties from a variety of feedstocks affect the way these flames burn, particularly at the conditions desirable for reduced emissions.
“The main trend so far has been toward electrifying cars and trucks, but aviation has been harder to electrify,” said Jacqueline Chen, a senior scientist at Sandia National Laboratories. “Rather, sustainable aviation fuel has emerged as the most attractive near-term option for decarbonizing aviation.”
One approach for aviation decarbonization involves adoption of so-called “drop-in” fuels – sustainably sourced bio-derived fuels that strictly meet the standards set for petroleum-based jet fuels and thus circumvent the need for fleet-wide recertification of engines currently flying commercially. But feedstocks vary, and the chemical composition of drop-in fuels can differ widely from those of fossil fuels. Low initial production yields for these experimental fuels tend to severely limit the testing required for full-scale engine certifications. High-fidelity simulations of these systems using exascale resources offers a promising route to understanding fundamental relationships between feedstock-dependent fuel properties and aircraft combustor performance – all while burning no physical fuel.
“The Combustion-Pele codes we’ve developed have proven to be flexible and adaptable to help us seamlessly shift focus from automobiles to aero engines,” Chen said. “They also include multi-physics treatments such as spray dynamics, soot formation, and thermal radiation that can be coupled with the underlying fluid dynamics and gas-phase chemistry.”
Frontier success
The Combustion-Pele codes have run calculations across three-fourths of Frontier’s more than 9,000 compute nodes, allowing the research team to simulate the direct injection and compression ignition inside a high-pressure diesel engine. These studies required 2.5 trillion degrees of freedom.
Frontier’s massive exascale throughput allowed researchers to capture the process from the cylinder diameter to flame ignition kernels smaller than a micron. The overall peak performance on Frontier reflects a 10-fold improvement over the best results on Summit in 2020 on 4,096 nodes. The same level of precision will now be applied to study aviation combustors.
“Frontier allows us the level of detail we need to look at the effect of using sustainable aviation fuel mixtures on combustion and soot formation,” said Marc Day, the Pele software architect and a computational scientist at the National Renewable Energy Laboratory. “We can gain this fundamental understanding of the physics and use that data with machine learning to develop predictive models that will help us test the next generation of jet fuels. These kinds of massive problems wouldn’t have fit on any computer before.”
Scientists on the Combustion-Pele project used Frontier to simulate the direct injection and compression ignition inside a high-pressure diesel engine in unprecedented detail. The team hopes to expand the simulation to capture the thousands of elementary reaction steps inside a jet engine.
Credit: Marc Day/NREL
What’s next?
The Combustion-Pele team hopes to expand the simulation to capture the thousands of elementary reaction steps in jet-engine chemical models, including those that govern flame propagation assisted by low-temperature ignition and its interaction with the complex flow of gases. This high-fidelity simulation data will be used to train deep neural networks and other types of models that could serve as the foundation for future simulation capabilities.
“The insights we gain can help us sift good from bad and narrow down development to a handful of the most promising fuel candidates,” Chen said. “That’s how we cut costs and accelerate production of drop-in fuels. Exascale can play a critical role in pushing these discoveries forward and enable the development of predictive models for use of sustainable aviation fuels.”
Support for this research came from the Exascale Computing Project, a collaborative effort of the DOE Office of Science and the National Nuclear Security Administration, from the DOE Vehicle Technologies Office, and from the DOE Office of Science’s Advanced Scientific Computing Research program. The OLCF is an Office of Science user facility.
UT-Battelle LLC manages ORNL for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. The Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit https://energy.gov/science.
