Visualization and Data Analysis
One of the principal missions of the OLCF is to help researchers gain a better understanding of their data through visualization and data analysis techniques. We seek out and engage with projects at ORNL and with collaborators who might benefit from applying visual data understanding techniques to scientific data and to find ways of doing visualization that are different, are more effective, and better integrate with other research activities at ORNL.
As scientific simulations scale up exponentially, so does the pressure on processes and tools for making use of the data. The OLCF is committed to developing tools that enable researchers to efficiently manage, analyze, and visualize simulation results.
OLCF users and the Scientific Computing liaisons have a number of tools at their disposal to help analyze data. Rhea is a cluster used for parallel analysis and remote visualization. The EVEREST visualization laboratory is also available at the OLCF, providing on-site users resources to visualize their scientific data created on the Titan supercomputer.
We support a variety of scientific visualization software, including Paraview and VisIt. This software is installed and supported across several of the OLCF computing platforms, included Titan and Rhea. Scientific liaisons at the OCLF often contributed solutions, code changes, bug fixes, and new features to these projects – often in direct response to our users’ needs.
The OLCF has also developed and deployed several parallel visualization and analysis tools that facilitate discovery in the EVEREST visualization laboratory. Additionally, both ParaviewVR and EnsightDR are supported in the EVEREST visualization laboratory for immersive analysis of data using positional tracking and stereo.
The OLCF has a dedicated liaison team to assist users in solving visualization and data-interpretation issues. Support can range from assisting users in analyzing data in the visualization laboratory to writing custom visualization tools for specialized needs to producing production-quality images and movies for publications and public relations.
Many routine data management tasks have been either automated or taken on by OLCF staff so that scientists can concentrate on their research. One of the goals is to ensure that calculations produce all the information the researchers need. One way of doing that is to generate more metadata, or self-describing in- formation, along with the scientific data. For example, a researcher might produce scores of binary files containing thousands of data points from calculations of different variables. If the researcher doesn’t keep detailed notes, it could be difficult later to match data sets with the calculations that produced them. To avoid such situations, OLCF staff can help researchers set up calculations so that they generate metadata to provide a context and description in every data file stored. Because generating metadata uses computer time, it is important to make the process as efficient as possible.
The workflow process has been automated to automatically archive data on tape as simulations run to avoid filling up the disk space on the supercomputers. The provenance information for the data—what is being archived, where the data are, which run they are from—is also preserved to provide a context for all data files.
The fast networks at the OLCF make it possible for off-site users to keep tabs on their simulations. Researchers can watch their work in progress via a Web browser anywhere they have Internet access. Faster input/output allows quick delivery and constant updates. The capability to monitor simulations in progress using the Internet enables users to catch mistakes and correct them early in a run before valuable processing time has been wasted. Faster networks also make it possible to move simulation results from the OLCF supercomputers to computers at a user’s own institution for processing. The increased portability of data allows users to get a head start on reviewing their results and react to interesting trends they observe as they analyze them.
For more information on OLCF Visualization and Data Analysis, contact Arjun Shankar, firstname.lastname@example.org