Titan Vaporizes R&D Time and Cost for Liquid Spray Studies

UTRC simulates hundreds of thousands of spray droplets in real time for potential jet engine and fire suppression applications

If you’ve ever gardened or watered a lawn, you know that pressing your thumb to a garden hose nozzle can transform a steady stream of water into a fanned spray.

The basic physics behind turning a jet of water into a fine mist also drives the complex process of converting liquid fuel into a spray for powering an aircraft engine.

Researchers at United Technologies Research Center (UTRC) are studying liquid atomization, or the breakup of liquid jets, to increase the efficiency of this process and thereby increase aircraft engine performance. Liquid atomization is also important in fire suppression systems such as chemical and water sprinklers and fire extinguishers.

UTRC is the corporate research arm of United Technologies Corp. (UTC), a provider of high-technology products and services to the building and aerospace industries. UTC businesses include Pratt & Whitney, Sikorsky, UTC Aerospace Systems, and UTC Building & Industrial Systems. To create technologies useful in several major industries, UTRC research teams tackle fundamental science and engineering problems that will impact multiple business unit product lines.

Because aircraft engine fuel needs to burn as fast as possible, UTRC research scientist Xiaoyi Li and fellow of research Marios Soteriou are interested in liquid jets that move a lot faster than the water from a garden hose—at about one-third the speed of sound. While their fundamental research begins with water, their results are applied to fuel and chemical jets.

Because these jets are difficult to analyze and measure experimentally, Li and Soteriou are simulating liquid atomization, casually known as “spray,” on the Titan supercomputer managed by the Oak Ridge Leadership Computing Facility (OLCF) at the Department of Energy’s (DOE’s) Oak Ridge National Laboratory. OLCF is a DOE Office of Science User Facility.

On Titan, they are testing the feasibility of using high-performance computing (HPC) to study spray formation. Their success is demonstrating that HPC modeling and simulation can provide insights and understanding not possible through experiments and can reduce the costs associated with physical prototyping.

Overcoming the R&D performance plateau

Engineers can reach a plateau at which they need new design or manufacturing methods to improve their products.

Thanks to decades of aerospace research and development (R&D), modern airplane engines are incredibly efficient. The liquid fuel that is injected into the engine is not wasted: almost all of it combusts. However, the fuel requires a certain amount of room to break up and burn, which increases engine size.

These engines are also remarkably safe. Prototypes are rigorously tested, and fueling systems are modified at any sign of instability. To build lighter planes that are more fuel-efficient, manufacturers want to build smaller engines; but that means fuel has to atomize more quickly so combustion can complete in a smaller space.

“Right now, engine manufacturers must estimate how much time they need to burn fuel effectively because they’ve about reached their limit of understanding of how to best mix everything based on experimentation,” Soteriou said.

But this design-build-test-rebuild method of product development is slow and expensive. UTC and its business units want to spend less time on prototype design and testing and build fewer costly prototypes, while still ensuring fuel spray and air are mixed effectively to avoid waste or mechanical instabilities.

“Engineers must design against combustion instabilities. Prototype engines, on occasion, exhibit unstable behavior. If the fuel is not mixing effectively with the flow of air coming into the combustor, combustion may either be incomplete; may cease completely, resulting in a phenomenon called blow-off; or may lead to large pressure oscillations in the combustor,” Soteriou said.

Soteriou and Li believe HPC simulations can help engineers better understand spray formation in smaller engines and lead to “virtual” prototypes that reduce the number of physical prototypes that must be built and tested.

“Computational models can substantially reduce the number of tests required to evaluate engine performance,” Soteriou said.

Individual tests can easily run in the hundreds of thousands of dollars, so the cost savings from simulation can be significant. Finally, there are environmental gains to be had, particularly for airplane engines.

“When you burn fuel, you get emissions of nitrogen oxides and carbon monoxide. By manipulating fuel injection, you can reduce these emissions,” Soteriou said. “And manufacturers must meet tougher emissions regulations, so this is becoming a requirement.”

The complexity of simulating sprays

Computational models can reduce R&D investments in time and cost, but there are several challenges to developing the high-fidelity, computational fluid dynamics (CFD) models needed to simulate fuel and chemical sprays.

Take aerospace applications, for example. In engines, spray atomization occurs in a small space rocked by turbulence and other aerodynamic forces. Liquid jets must be simulated across a range of spatial and temporal scales that require calculations over many orders of magnitude, from microseconds to tens of milliseconds, and from micrometers to centimeters.

On top of spatial and temporal scales, researchers need to accurately reproduce the sharp variation of fluid properties across the liquid-gas interface. Of particular concern is the density variation, which can cover orders of magnitude and lead to destabilization of the computation—unless advanced models and substantial computational resources are used. Finally, leading CFD simulations need to simulate all these parameters under realistic operating conditions, such as the high temperatures at which spray atomizes in a jet engine.

To meet and exceed these requirements, the UTRC team developed a unique computational model that uses advanced numerical algorithms and multigrid methods to simulate spray atomization under a number of conditions at a high density ratio. Then, to ensure their CFD model matched real-world conditions, Li and Soteriou searched for a detailed experimental study to simulate and compare the accuracy of their model predictions.

Blink and you could miss it

To measure spray experimentally, researchers need to observe liquid jets in microsecond time steps and measure droplets from every angle.

“People can use lasers to make spray measurements, but this is an expensive approach and is less accurate in the region of spray formation where the laser light is obscured by the dense liquid. Alternatively, intrusive instruments can be used to study spray experimentally, yet placing things in the flow path can impact the liquid breakup behavior so one may not be measuring what is actually happening in the real device,” Soteriou said. “Finally, almost all experimental measurements are performed at ambient conditions, or regular temperature and pressure, but engines operate at high pressures and temperatures. Because experiments under these conditions are extremely expensive, in addition to being difficult to measure, they’re rarely undertaken. However, this is the information we need to adjust our engine designs to achieve greater performance.”

Li and Soteriou turned to a 2004 experiment in which researchers at Oklahoma State University and the University of Michigan used shadowgraphy (capturing many layers of liquid in an image) and holography (capturing droplets in three dimensions) to record images of droplets and the ligaments that connect droplets before they break apart.

“If you took a picture of this with your regular camera, you’d just see a cloud,” Soteriou said.

Not the case here. The study’s cameras were set at 7 nanosecond pulses and could change position within 1,000 nanometer accuracy.

“This was a very detailed study that measured many near-field details. It’s not the kind of experiment that can be repeated often,” Soteriou said. “We knew if we could replicate their results using HPC simulation, we would validate simulation as a substitute that could accurately and more cost-effectively analyze spray formation.”

After choosing this experimental validation study, Li and Soteriou faced another challenge: finding a computer that could enable such a large-scale simulation.

“UTRC has computer clusters, but the massive calculations needed here would more than saturate our computational resources,” Soteriou said. “We needed access to a much larger system to show what is possible.”

Titan vaporizes the competition

At a peak performance of 27 petaflops (or 27 quadrillion operations per second), Titan was the right system to simulate spray in new detail. UTRC had already used Titan’s predecessor, Jaguar. Through the OLCF industrial HPC partnership program Accelerating Competitiveness through Computational Excellence (ACCEL), Soteriou and Li received a Director’s Discretionary allocation to complete an earlier study on spray atomization. That experience helped them to successfully compete for a grant of 20 million hours on Titan through DOE’s Advanced Scientific Computing Research Leadership Computing Challenge.

“Our first objective was to demonstrate that our CFD model when executed on Titan can reproduce real life by reproducing the experimental results,” Soteriou said. “The second objective was to use the high resolution results enabled by Titan to understand the physical mechanisms by which the liquid breaks apart. The third objective was to use our simulations on Titan to identify the computational weaknesses in our model so we can enhance our computational tool to make it faster down the road.”

Using the high-fidelity, multiple-injection simulation code developed by UTRC, they tested their computational model against the experimental data collected using advanced camera techniques. By further refining these simulations, Li and Soteriou were able to go beyond experimental capabilities to model a hard-to-observe region of spray that is important to equipment design.

“Quantifying spray is particularly difficult very close to the injector, where measurements are more obscured by the liquid,” Soteriou said. “People can measure downstream, but by the time the spray arrives downstream, it’s too late—you don’t really know what happened in the injector, and so you can only guess how to change it to improve the downstream distribution. As a result, the design of fuel injectors has been very empirical. Computational modeling is really the only way we have today to look at the near-field of the spray.”

Using a technique known as adaptive mesh refinement on Titan, Li and Soteriou simulated hundreds of thousands of droplets by creating a very fine grid near the nozzle generating the spray. To conserve computing power for the challenging near-field simulation while still generating a complete model of the spray evolution, they used a coarser grid farther away from the nozzle where droplets are effectively separated. This approach was validated by comparing the results of the adaptive mesh refinement simulation with calculations in which the grid was uniform throughout. Such a calculation was possible only because of the computational resources Titan could deliver.

“With these computational tools, for the first time we can actually see what happens in the near-field,” Soteriou said.

For the team’s uniform grid simulation, in which the finest grid is spread over the whole domain to study detailed liquid atomization as well as gas flow, the resolution equaled 503 million grid cells.

“Our 16-day simulation would have taken approximately 3 years on our internal systems,” Soteriou said. “Titan made an intractable problem doable.”

Li added, “We can say we are the first to simulate realistic conditions for this kind of spray simulation,”

Not only did Titan enable the UTRC team to show in simulation what can’t be seen experimentally, but also their in-house code successfully ran on Titan’s system and simulated a complex 1,000:1 density ratio between gas and liquid.

“It’s difficult to simulate the very sharp interface between liquid and gas accurately when there is such a high density ratio across it,” Soteriou said. “But Titan’s horsepower enabled us to do this.”

Li and Soteriou documented preliminary results from this study in a conference paper presented at the 50th Aerospace Sciences Meeting of the American Institute of Aeronautics and Astronautics; a more recent conference paper for the 26th annual conference of the Institute of Liquid Atomization and Spray Systems presents more extensive results and analyses.

“We want to use these results to demonstrate to our company leadership the cost benefits of using larger-scale modeling and simulation,” Soteriou said. “Titan allows us to show what is possible, and as a result, it motivates growth of our internal HPC resources, too.”

Oak Ridge National Laboratory is supported by the US Department of Energy’s Office of Science. The single largest support of basic research in the physical sciences in the United States, the Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

Related publications:

1. Li, X, and Soteriou, MC, “High-fidelity Simulation of High Density-Ratio Liquid Jet Atomization in Crossflow with Experimental Validation,” Institute for Liquid Atomization and Spray Studies-Americas 26th Annual Conference on Liquid Atomization and Spray Systems, Portland, Oregon, May 2014. Available at ilass.org/2/conferencepapers/54_2014.pdf.
2. Li, X, and Soteriou, MC, “Prediction of High Density-Ratio Liquid Jet Atomization in Crossflow Using High Fidelity Simulations on HPC,” American Institute of Aeronautics and Astronautics (AIAA)-2012-0175, 50th AIAA Aerospace Sciences Meeting, Nashville, Tennessee, January 2012. Available at https://arc.aiaa.org/doi/abs/10.2514/6.2012-175.