Researchers are taking 3D holograms beyond science fiction theatrics with recent advances in augmented reality (AR) techniques.
During their summer internships at the US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL), Cooper Colglazier and Jesse Vomfell investigated how AR could be effectively applied to study various aspects of the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF), a DOE Office of Science User Facility located at ORNL.
Unlike virtual reality, which immerses viewers in an artificial setting entirely separate from their actual surroundings, AR augments existing environments with virtual data, typically via holograms overlaid onto physical structures. This way, users can interact with tangible and intangible information simultaneously to obtain a more complete understanding of the object in question.
Colglazier and Vomfell developed source code containing best practices, as well as specific hardware and software components, required to develop an AR tool for monitoring Summit. Additionally, they wrote a technical report describing obstacles they encountered and the solutions they devised in response. These resources provide a foundation for the future production of such a device. Both interns participated in the Higher Education Research Experiences program under the mentorship of Computer Scientist Jamison Daniel, an R&D staff member in the OLCF’s Advanced Data and Workflow Group.
OLCF interns Cooper Colglazier (left) and Jesse Vomfell (right) collaborated on a summer research project to determine the feasibility of using AR to study Summit and lay the groundwork for a new resource that could help scientists, engineers, and maintenance staff identify and resolve issues more efficiently than what is possible with conventional monitoring methods.
“They leveraged a lot of existing technologies, but these students created all of the source code from the ground up,” Daniel said. “They considered the constraints and benefits of emerging AR capabilities and deployed them in the Summit machine room to orchestrate real-time data analysis.”
After learning to code in the C# programming language, the interns used the Unity game engine—a software framework traditionally associated with video game design—to interpret detailed data regarding the supercomputer’s cabinets and the machine room floor plan. Using this information, they designed a prototype to test on Summit.
“We developed a prototype geared toward maintenance personnel that would allow them to assess the machine in real time,” said Vomfell, an undergraduate student studying computer science at the University of Tennessee, Knoxville. “With this application, they could measure aspects of Summit such as temperature and GPU load.”
The tool is compatible with an AR headset called the Microsoft HoloLens and has two distinct modes. The machine room mode covers Summit with a holographic replica of the supercomputer to help ORNL staff identify and address maintenance issues onsite.
“Mapping virtual data to a physical source leads to a better spatial understanding of the machine,” said Colglazier, a graduate student studying human–computer interaction at Georgia Tech. “Looking at Summit through the HoloLens provides insights into the performance of the individual nodes inside each cabinet.”
For users unable to access the machine room directly, the overview mode allows them to remotely monitor the machine by inspecting a miniature AR model of Summit and marking areas of concern for others onsite to examine.
“The ideal AR tool would be optimized not only for people at the lab, but also for research teams from academia and industry who want to observe how their codes are running on Summit,” Vomfell said.
Together, these options could eventually enable more mobile and dynamic workspaces that streamline everyday activities for scientists and engineers who work on Summit. Because the interns’ prototype demonstrated a promising use of AR, they hope their results stimulate similar efforts at other facilities across ORNL.
“Essentially, we wanted to determine how AR could be used in future workflows at the lab,” Colglazier said. “The benefit of AR is that you are not tied to the hardware and can see all kinds of data visualizations without changing the setup of the room.”
According to Daniel, AR could also potentially provide a cost-efficient, portable alternative to large-scale visualization laboratories that require mounted cameras, tracking markers, and other supplementary equipment. These types of facilities often only accommodate one person at a time and require users to remain in a designated area to avoid image distortion, but AR has no such limitations.
“Think of those elaborate sidewalk chalk illustrations that trick your eyes into thinking you’re about to fall off a cliff or run into a wall,” Daniel said. “AR does something similar, but imagine that, when you move, the artist can immediately erase the entire image and redraw it to perfectly fit your new vantage point.”
Although the students designed their monitoring tool to study temperature and GPU load, they stressed that future projects would require additional research to determine which variables instead of or in addition to these examples would be most useful to maintenance staff. Going forward, researchers can save time and cost by referring to the interns’ source code and report for guidance and incorporating new features into their tool’s existing framework.
During their 3 months at ORNL, the interns also produced a video showcasing their work and furthered their career development by practicing technical skills and collaborating with experienced staff members. Both students have worked on other AR projects since returning to their universities and hope to gain additional expertise on the subject.
ORNL is managed by UT-Battelle for the Department of Energy’s Office of Science, the single largest supporter of basic research for the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit https://science.energy.gov.
