Chillers at the OLCF help keep supercomputers running efficiently by cooling water to 42 degrees. Recent findings indicate the OLCF can raise the temperature of water running through Titan to 48 degrees.
Engineering firm’s study to guide decision making for keeping Titan running efficiently
Staff members at the US Department of Energy’s (DOE’s) Oak Ridge National Laboratory are committed to performing world-class science at the lowest possible environmental cost.
In the spirit of saving energy, Jim Rogers, Director of Computing and Facilities for ORNL’s National Center for Computational Sciences, brought in an engineering team to evaluate how existing supercomputers could be cooled more efficiently.
Titan, the flagship supercomputer at the Oak Ridge Leadership Computing Facility (OLCF)—a DOE Office of Science User Facility—is a Cray XK7 capable of 27 petaflops, or 27 quadrillion calculations per second. Titan is the most current generation of OLCF supercomputers and has been cooled in the same manner as its predecessors—via chillers that cool around 2,200 gallons of water per minute to 42 degrees Fahrenheit. Heat is ejected from a secondary loop exclusive to the XK7 to the chilled water loop. The water then returns to the chillers to be cooled again, starting the process over.
There is a direct correlation between the temperature of the supply water and the energy to cool it to the correct temperature. Colder supply temperatures have higher operational costs. Titan currently operates at 42 degrees. However, recent work indicates that the OLCF can raise the supply temperature without negatively influencing hardware reliability. In addition, operating costs will decrease. The OLCF anticipates it will be able to change the water temperature to 48 degrees, and though a six-degree change sounds small, it potentially could save the OLCF hundreds of thousands of dollars over the remainder of the system’s operational life.
Temperature alone is not the only characteristic that determines cooling efficiency, though. Rogers said there are strong indications that increasing the water’s flow rate at the cooling towers can increase the operating efficiency under stressful environmental conditions including high wet bulb (the temperature air would have if it were cooled to the saturation point).
The OLCF will run a variety of combinations of water temperatures and flow rates over the coming months to identify the best balance of performance, energy efficiency, and cost savings. By the end of fiscal year 2015, OLCF staff members expect to have a set of revised operational policies in place for managing water supply temperatures and flow rates that further increase operational reliability and reduce cost.
Oak Ridge National Laboratory is supported by the US Department of Energy’s Office of Science. The single largest support of basic research in the physical sciences in the United States, the Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.
