Tool developers create new technologies to help Titan users
Nearly two dozen software tool developers gathered recently at ORNL to boost the integration of the award-winning Adaptable I/O System for Big Data (ADIOS) into their technologies.
The ADIOS Code Sprint was held August 19–20, bringing 22 developers together from research organizations across the United States, including ORNL, Lawrence Berkeley National Laboratory, Sandia National Laboratories, the University of Oregon, Kitware, North Carolina State University, Rutgers University, Georgia Tech, and the University of California–Davis.
By bypassing the disc storage medium, ADIOS enables researchers to analyze simulation data much more efficiently and was named one of R&D Magazine’s Top 100 Technologies in 2013.
“The goal of the ADIOS Code Sprint was to have tool developers sit in one room together, give presentations, talk to each other and integrate ADIOS into their technologies, so that application developers can benefit from ADIOS much more easily,” said Gary Liu, a research scientist with the Scientific Data Group in ORNL’s Computer Science and Mathematics Division and a leading developer of ADIOS.
The researchers were separated into three groups with one handing off data to the next group, hence the phrase “Code Sprint.”
The first stage, and the starting point of the sprint, looked at the I/O stream—the raw simulation data that ADIOS developers are working to manipulate—and how it can be re-organized or reframed to optimize the movement of data from input to output.
The next stage was not only to take the I/O stream and perform intermediate data transformations—a class of technologies that change the format/encoding of data to optimize it, such as indexing and compression—but also to provide a framework for these transformations that gives user support within ADIOS itself.
“This framework would benefit Titan researchers in a number of ways, such as speeding up simulation I/O times, reducing storage space, and accelerating reads for analysis, which all come down to less down time and more research time for Titan users,” said LBNL computer scientist John Wu.
The final job and lap was the visual representation of data. Once the I/O and intermediate teams were satisfied with their data, it was the job of the visualization team to produce meaningful graphics. By using the ADIOS data streaming capability, according to Ken Moreland of SNL, researchers can produce a wide range of visual representations of data, from simple 2-dimensional graphs to more complex 3-dimensional graphs, with significantly reduced I/O overhead.
“In the past, we demonstrated how to use ADIOS as a remote, interactive visualization tool with the data of a running simulation,” said Norbert Podhorszki of ORNL’s Scientific Data Group, an ADIOS developer and a liaison to application developers. “With the sprint, we taught other tool developers how to create generic solutions so that anyone can use ADIOS for similar purposes.”
The sprint event was co-sponsored by the OLCF and Scalable Data Management, Analysis, and Visualization, an institute in the Department of Energy’s Scientific Discovery through Advanced Computing program.—by Austin Koenig
