Tom Papatheodore leads Summit user training

Tom Papatheodore, an OLCF high-performance computing engineer and member of the User Assistance and Outreach Group, connects users to the resources and training materials they need to do cutting-edge computational science. “A big part of the reason I enjoy my job is because I get to be a part of the teaching and training,” he said.

The Faces of Summit series shares stories of the people behind America’s top supercomputer for open science, the Oak Ridge Leadership Computing Facility’s Summit. The IBM AC922 machine launched in June 2018.

Tom Papatheodore doesn’t have all the answers for how to run cutting-edge science at the Oak Ridge Leadership Computing Facility (OLCF), but he can likely identify the experts who do.

As the lead OLCF user training organizer, Papatheodore sits at the confluence of people who use supercomputers and people who make supercomputers usable. With the first user programs for Summit set to begin in January 2019, he’s currently focused on ensuring that users experience a smooth start to the newest chapter of leadership-class computing at the OLCF, a US Department of Energy (DOE) Office of Science User Facility located at DOE’s Oak Ridge National Laboratory (ORNL).

“There are definitely some differences between Summit and Titan, the OLCF’s previous leadership-class system, that users will need to be aware of,” said Papatheodore, an OLCF high-performance computing (HPC) engineer and member of the User Assistance and Outreach (UAO) Group. “A big part of the reason I enjoy my job is because I get to be a part of the teaching and training.”

In October, Papatheodore organized the OLCF’s annual GPU hackathon, the finale of a larger series in 2018 where local and visiting teams worked with experts to update their codes for GPU-accelerated computing. The OLCF-hosted event marked the first time hackathon teams had access to an 18-node auxiliary system called Ascent that contains the same node-level architecture as Summit.

The hackathon series is just one of several ways the OLCF supports scientific application developers each year. Additionally, Papatheodore contributes to more traditional training and user support, such as hands-on workshops, webinars, documentation, and troubleshooting, to help users maximize their time on OLCF systems.

To connect users with the resources they need to do computational science, Papatheodore relies on his own HPC experience, his enthusiasm for sharing knowledge, and his fondness for solving problems.

Bigfoot to Big Bang

While growing up in Michigan, Papatheodore always felt drawn to the seemingly inexplicable—magicians’ optical illusions and television tales of alien encounters. “When you get older, you realize none of that stuff is real,” he said.

Science presented equally compelling phenomena—an entire universe filled with exploding stars, black holes, and undetected planets. Furthermore, popular science media such as Carl Sagan’s Cosmos series or Stephen Hawking’s A Brief History of Time backed up these astounding claims with empirical evidence. “That personally made me interested,” Papatheodore said.

He followed that interest at the University of Michigan–Flint, where he studied physics and astrophysics. Though he claims he’s never had a great aptitude for advanced math, Papatheodore’s pursuit of scientific knowledge supplied ample motivation to learn each new concept.

“It’s like figuring out puzzles,” he said. “When you do it one step at a time, it all builds on itself. It becomes possible.”

After enrolling in the theoretical and computational astrophysics PhD program at the University of Tennessee, Knoxville (UTK), Papatheodore encountered his biggest puzzle yet: HPC.

Because many stellar objects can be studied only by observing the night sky, scientists have turned to computer simulation to better understand the physics of stars, galaxies, and other cosmological phenomena.

In graduate school, Papatheodore learned to program for HPC while he was part of a team dedicated to modeling and simulating Type Ia supernovae—exploding white dwarf stars that are also used as reliable distance markers for objects beyond our local group of galaxies. Specifically, he used a programming model called OpenMP to speed up the calculations of a nuclear reaction network called XNet that is used as part of the team’s FLASH software. The updated code was later used by a group of UTK and ORNL researchers to produce high-fidelity supernovae simulations on the OLCF’s Jaguar and Titan supercomputers.

“I’d taken a couple programming classes before graduate school, but all of a sudden I was being thrown into the deep end of the pool with a big HPC code,” Papatheodore said. “Fortunately, I was working with a well-written, well-documented code, and I had people to learn alongside.”

In addition to learning new skills, he also had the chance to teach concepts he’d previously mastered, lecturing and guiding students in introductory physics classes at UT.

“I like teaching people. I like getting to go out and interact with people and see how they are using these large systems.”

Training for Summit

Upon receiving his PhD in 2015, Papatheodore became an ORNL postdoctoral associate under the Computational Scientists for Energy Environment and National Security (CSEEN) program. Tasked with updating parts of FLASH in advance of the Summit supercomputer’s arrival, Papatheodore gained his first experience programming for GPUs. As part of this work, he ported the same nuclear reaction network he worked on as a graduate student to run on the SummitDev system, a development system meant to bridge the gap between Titan and Summit.

Though he had moved on from the project by the time the new version of FLASH ran on the Summit supercomputer for the first time in early 2018, Papatheodore’s CSEEN experience reinforced the value of using state-of-the-art technology to advance scientific discovery. “It’s neat to be able to use technology like GPUs to be able to do things that couldn’t have been done before,” he said.

In his current position at the OLCF, Papatheodore helps others have that same sense of accomplishment.

With Summit being readied for full production, the fun for users is just getting started. In early November, Papatheodore helped coordinate a webinar covering multi-GPU programming on Summit. During the next training event in early December, Papatheodore and his OLCF colleagues will be on hand alongside technology vendors for a 4-day workshop dedicated exclusively to helping users get started on Summit. The workshop will provide attendees with a comprehensive overview of how to navigate the IBM AC922 system. Additional training events are being planned for 2019.

Whatever problems users encounter along the way to breakthrough science, Papatheodore said he will be ready and willing to assist in any way he can.

“I like teaching people,” Papatheodore said. “I like getting to go out and interact with people and see how they are using these large systems.”


ORNL is managed by UT-Battelle, LLC for the Department of Energy’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit