The “Pioneering Frontier” series features stories profiling the many talented ORNL employees behind the construction and operation of the OLCF’s exascale supercomputer, Frontier. The HPE Cray system will be available to users later this year.
The world’s first exascale supercomputer, Frontier, will soon lead users in the direction of discovery—and Justin Whitt has been holding the compass since its inception. As Frontier’s project director for the past 5 years, Whitt has overseen nearly every detail required to deliver and site the new, groundbreaking system.
During the first part of this mammoth undertaking, Whitt’s job included overseeing the Frontier data center’s construction and site preparation along with the installation of the infrastructure required to support Frontier prior to its arrival at the US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL). Beyond the logistics of construction, his role also involved working with Frontier vendors HPE and AMD, along with ORNL staff, to develop the hardware, software, and scientific applications for the system.
Frontier Project Director Justin Whitt sits in the supercomputer’s data center, entirely rebuilt from below the floor up to accommodate Frontier’s electrical and cooling needs. Image: Carlos Jones/ORNL.
As the first of more DOE exascale supercomputers to come, the HPE Cray EX Frontier is capable of 2 quintillion calculations per second, or 2 exaflops. If every person in the world completed one calculation every second, it would still take the entire global population more than 4 years to do what Frontier can do in one second. The system was built by global enterprise technology leader Hewlett Packard Enterprise, or HPE Cray, and features next-generation, leading-edge components from AMD. Already ranked the world’s fastest supercomputer by the TOP500 list, Frontier will be available for user operations later in 2022.
“With Frontier, scientists will be able to pack more physics and more parameters into their computational simulations, as well as study new phenomena in various fields, such as biology, fusion, engineering, chemistry, physics, and others,” said Whitt, who also serves as the OLCF’s program director. “This will lead to higher-resolution solutions at even greater fidelity, which will allow researchers to gain insights into problems that were previously impossible to study.”
After Whitt earned a graduate degree in computational engineering in 2009, he received an opportunity to come to the laboratory as a computational scientist with the National Institute for Computational Sciences (NICS) at the University of Tennessee, Knoxville (UTK). At NICS, he began working with the Kraken supercomputer, a Cray system that was capable of 831 teraflops and housed at ORNL. He eventually became the chief project officer and then acting executive director for the Joint Institute for Computational Sciences, a facility that is home to ORNL’s core partnerships with UTK. In 2015, Whitt was offered the position of deputy director of the Oak Ridge Leadership Computing Facility (OLCF), a DOE Office of Science user facility located at ORNL.
“I jumped at the chance to work with the highly talented folks at the OLCF,” Whitt said. “NICS was a sister organization to the OLCF, and it was set up to look identical, as far as how it operated. That made it easy. In some ways, I just moved one building over, essentially.”
When Whitt began working at the OLCF, his team started working with DOE headquarters to determine the needs of the next compute system at ORNL.
“At the time, our systems were four to five times oversubscribed,” Whitt said. “The systems were unable to fill all the needs of the scientists, both in terms of how they wanted to use them and how capable the systems were.”
Ironically, Whitt said, the process of installing a supercomputer roughly follows the scientific method. First, needs are observed, and then hypotheses about the different ways of meeting those needs are formed. “Then we complete an alternative analysis to test each one, so to speak,” Whitt said.
After the system vendors were selected, work on the system began.
One of the largest initial projects, Whitt said, was getting the power and cooling ready for Frontier.
“The power was a larger amount of work than we’ve ever taken on before—bringing new power lines down across campus to the building,” Whitt said. “The amount of construction we’ve done for Frontier is above and beyond what we’ve had to do in the past.”
The most challenging part of the project, Whitt said, has been the ongoing COVID-19 pandemic. Whitt has worked with headquarters, suppliers, and vendors to mitigate issues that have arisen during the process of building Frontier.
“It’s touched everything,” Whitt said. “From getting transformers on site for the construction piece of the project to getting the parts needed to build the computer, the supply chain impacts from COVID-19 have been dramatic.”
Whether it has been navigating around supply chain issues or helping suppliers and vendors get their parts qualified quickly, Whitt and his team worked tirelessly on the project during this unprecedented event.
Entering a new Frontier: just one section of the HPE Cray EX exascale system at ORNL. Image: Carlos Jones/ORNL.
Meanwhile, as the OLCF program director, Whitt is also responsible for operating OLCF systems and fulfilling the potential of the Advanced Scientific Computing Research, or ASCR, user programs. Whitt also works closely with the Exascale Computing Project (ECP), a DOE organization that has been preparing software to run on the upcoming exascale systems. ECP Project Director Doug Kothe and Whitt work to ensure that the ECP and Frontier projects are coordinated and marching in step toward the advancement of DOE’s exascale mission.
As for the science he’s most excited about seeing on Frontier, computational fluid dynamics (CFD), or the modeling of fluid flows, tops the list.
“I’m a CFD person by training, so I have a soft spot in my heart for CFD problems. I’m drawn to those,” he said.
He also said the application of artificial intelligence (AI) will be a big draw for researchers looking to train machine learning algorithms on large datasets. These researchers will be able to make inferences that human beings would never be capable of making, leading to new discoveries in a variety of fields.
“Frontier’s sheer power will allow researchers to find patterns, make inferences, and make sense of massive amounts of data,” Whitt said. “In essence, Frontier will provide a first-of-its-kind capability for AI, which will power many new AI applications on the system.”
When asked about his favorite aspect of the project, Whitt didn’t hesitate to say, “The people.”
“Frontier touches a lot of people, and a lot of people touch Frontier. It would be wild to sit down and count up all the different people it takes to deploy the largest supercomputer in the world.”
OLCF Program Director Justin Whitt surveys the cabinets of the Frontier supercomputer. Image: Carlos Jones/ORNL.
ORNL’s initial request for proposals to build Frontier brought together more than 150 people from all over the DOE complex and comprised seven different national laboratories. These personnel looked at the various offerings and provided input and expert opinions on the various technology offerings. Since then, an army of construction workers, electricians, engineers, computer scientists, and researchers have worked in concert to create the hardware and software for Frontier and the data center to house it.
“The partnerships span DOE and private industry. There are not many places in the world where this can be done. It’s awe-inspiring, the number of gifted people that have dedicated their time to make this happen,” Whitt said.
UT-Battelle LLC manages Oak Ridge National Laboratory for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, visit https://energy.gov/science.
