The Adaptable IO System (ADIOS) provides a simple, flexible way for scientists to describe the data in their code that may need to be written, read, or processed outside of the running simulation. By providing an external to the code XML file describing the various elements, their types, and how you wish to process them this run, the routines in the host code (either Fortran or C) can transparently change how they process the data.

The in code IO routines were modeled after standard Fortran POSIX IO routines for simplicity and clarity. The additional complexity including organization into hierarchies, data type specifications, process grouping, and how to process the data is stored in an XML file that is read once on code startup. Based on the settings in this XML file, the data will be processed differently. For example, you could select MPI individual IO, MPI colletive IO, POSIX IO, an asynchronous IO technique, visualization engine, or even NULL for no output and cause the code to process the data differently without having to either change the source code or even recompile.

The real goal of this system is to give a level of adaptability such that the scientist can change how the IO in their code works simply by changing a single entry in the XML file and restarting the code. The ability to control at a per element basis and not just a data grouping such as a restart, diagnostic output, or analysis output makes this approach very flexible. Along with this detail level, a user can also just change which transport method is used for a data type such as a restart, analysis, or diagnostic write.

For the transport method implementer, the system provides a series of standard function calls to encode/decode data in the standardized .bp file format as well as “interactive” processing of the data by providing direct downcalls into the implementation for each data item written and also callbacks when processing a dodata stream once a data item has been identified along with its dimensions and a second callback once the data has been read giving the implementation the option to allocate memory and process the data as close to the data source as is reasonable.



Current stable version: ADIOS 1.12.0 (Download 1.12.0)

What’s new in 1.12.0

  • New chapter in user manual on using ADIOS with the burst buffer of the Summit@OLCF and Cori@NERSC supercomputers, see Chapter 16.
  • SZ lossy compression, see for details. Options in ADIOS are discussed in the manual Chapter 8.
  • LZ4 compression was added by a user of ADIOS, René Widera from HZDR, Germany
  • Profiling API was added by Kevin Huck of the TAU team Tool developers need to use the adiost_callback_api.h. A basic default tool implementation is in src/core/adiost_default_tool.c
  • The FLEXPATH staging method from Georgia Tech has been redesigned for more robust and faster data staging. The new version of the chaos library is required, see section in the user manual.
  • Transport parameter “local-fs=1” will allow the POSIX and MPI_AGGREGATE methods write the output to a distributed file system. The output path is the same on all compute nodes. Any reader process will see the global arrays (definition) but only can successfully read that portion of the data that was written on the same compute node. Think of checkpoint/restart as an example.
  • Bug fixes for time-aggregation, reading >2GB blocks from file, CMake build, etc.

Requirements and optional features

  • Python (required)
    The XML processing utility utils/gpp/ is a code written in python using xml.dom.minidom. It is used to generate C or Fortran code from the XML configuration files that can be included in the application source code. Examples and tests will not build without Python.
  • MPI and MPI-IO is required to build the parallel library, but a sequential-only library can be built by configuring with the –without-mpi option.
  • Mini-XML: ADIOS depends on and it includes the Mini-XML library but it can be built separately if necessary. It can be downloaded from
  • A Fortran 90 compiler (optional)
    The Fortran API is built only if a Fortran 90 compiler is available.
    Use --disable-fortran at configuration time to disable this feature.
  • DataSpaces staging (optional)
    • The DataSpaces model provides a separate server running on separate compute nodes, into/from which data can be written/read with a geometrical (3D) abstraction. It is an efficient way to stage data from one application to another in an asynchronous (and very fast) way. Multiple steps of data outputs can be stored, limited only by the available memory. DataSpaces can be downloaded from
    • DataSpaces is supported on Cray Gemini, Portals and Infiniband networks.
  • Flexpath staging (optional)
    • Flexpath is an asynchronous data transport method built to ensure scalable I/O through the use of staging areas. It is built on top of the EVPath event-driven messaging library, which allows for the creation of arbitrary network overlays. Flexpath also works on top of several popular high-end network interfaces, such as Infiniband, Portals, and Gemini.
    • EVPath acquisition instructions are at the EVPath web site, or in the ADIOS manual under the installation guide.
  • FastBit indexing library (optional)
  • Alacrity indexing library (optional)
  • Numpy (optional)
    • If you have Numpy, you can build ADIOS read/write modules for Numpy separatly after building ADIOS. See details in the Installation section of the User’s Manual.
  • Matlab (optional)
    • If you have Matlab, you can build the ADIOS reader for Matlab separatly after building ADIOS. See details in the Matlab section of the INSTALL file before configuring adios.
  • HDF5 (optional)
    • The bp2h5 converter utility to HDF5 format is built only if a HDF5 library is available. Currently ADIOS uses the 1.6 version of the HDF5 API but it can be built and used with the 1.8.x version of the HDF5 library too. Use the option --with hdf5= to configure ADIOS.
    • HDF5 can be downloaded from
  • NetCDF (optional)
    • The bp2ncd converter utility to NetCDF format is built only if NetCDF is available. ADIOS can use a sequential build of the NetCDF-3 or NetCDF-4 library. Use the option –with-netcdf= or ensure that the NETCDF_DIR environment variable is set before configuring ADIOS.
    • NetCDF version 4.3.2 or the legacy version 3.6.2 can be downloaded from
  • Parallel HDF5 (optional)
    • The transport method writing files in the Parallel HDF5 format is built only if a parallel version of the HDF5 library is available. You need to use the option with phdf5= to build this transport method.If you define Parallel HDF5 and do not define serial HDF5, then bp2h5 will be built with the parallel library.Note that if you build this transport method, ADIOS will depend on PHDF5 when you link any application with ADIOS even if you the application does not intend to use this method.If you have problems compiling ADIOS with PHDF5 due to missing flags or libraries, you can define them using
      • --with-phdf5-incdir= ,
      • --with-phdf5-libdir= and
      • --with-phdf5-libs=
  • NetCDF-4 Parallel (optional)

Revision History

Revision History

1.11.0 Release

  • Time aggregation (temporal aggregation of an output in memory) see adios_set_time_aggregation() or the <time-aggregation> element in the XML syntax. (Manual Chapter 7)
  • ZFP lossy compression transform method (Manual Chapter 8)
  • Python wrapper includes functions for: (Manual section 14.2)
    • Selecting transforms and time aggregation: adios_set_transform()
    • Time aggregation: adios_set_time_aggregation()
    • Set maximum buffer size used by any ADIOS group: adios_set_max_buffer_size()
  • Collect min/max statistics only by default (Manual section 4.1.3)
    adios_declare_group() last argument type changed to be an option for statistics.
    Options are: adios_stat_no, adios_stat_minmax, adios_stat_full, and adios_stat_default, which is minmax
  • Added functions to C API to detect available methods in the ADIOS installation
    • adios.h: adios_available_write_methods()
    • adios_read.h: adios_available_read_methods()
    • adios_transform_methods.h: adios_available_transform_methods()
    • adios_query.h: adios_available_query_methods()
  • Bug fixes
    • Performance bug in MPI_AGGREGATE method in 1.9/1.10 fixed
    • fix: bug building with hdf5 1.10

1.10.0 Release 2016

  • Updated Query API (Manual Chapter 9)
  • Minmax, FastBit and Alacrity query methods
  • adios_group_size() is optional
  • ADIOS builds without first installing Mini-XML separately
  • bprecover utility to recover datasets with many output steps where a step becomes corrupted (Manual section 10.6)
  • Point selections can provide a container selection to improve read performance
  • Added –without-mpi option to configure, so that only the sequential libraries are built
  • Adios Python wrapper
    • Updated to support both python 2 and python 3
    • Added read options with point and block selection
    • Added group management on reading
    • Updates on auto completion with ipython
  • Bug fixes
    • Build on OS X, both clang and gcc supported
    • Better xml processing to allow for multiple text lines as parameters for a method
    • Support adios_inq_var_stat() when reading a file in streaming mode
    • bpmeta does not skip any subfiles anymore when used with threads

1.9.0 Release Dec 2015

  • Array attributes are supported
    • e.g axes = {“X”,”Y”,”Z”}
  • adios_define_attribute_byvalue()
    • to define scalar attributes with floating point variables instead of strings for precision
  • Update mode when appending to a file
    • to add variables to last timestep instead of a new one
  • Python/Numpy wrapper improvements:
    • Numpy-style array notations
      e.g, var[1:5, 2:10], var[1:5. :], var[:5,…]
    • Support for ADIOS write APIs
    • Hint/docstring support
    • Support for pip install and update
  • Added adios_version.h to provide release and file format versions
  • Bug fixes
    • fix: memory leak in POSIX method
    • fix: adios_write() now accepts const * void data from C++ apps
    • fix: Cray compiler support
    • fix: reading of compressed, zero size arrays on some processes
    • fix: scaling bugs in aggregate method writing > 2GB per process or aggregating data into a file over 4GB

1.8.0 Release Dec 2014

  • Query API
    • extends the read API with queries
  • Staging over WAN (wide-area-network)
    • ICEE method (requires FLEXPATH)
  • skeldump utility
    • to generate info and code from output data to replay the I/O pattern
  • bpmeta utility
    • generates metadata file (.bp) separately after writing the data using MPI_AGGREGATE method with metadata writing turned off
  • I/O timing statistics and timing events can be collected
  • New stage writer code for staged I/O, where output data (list of variables and their sizes) is changing at every timestep. See examples/stage_write_varying
  • fix: staging with multiple streams allowed
  • fix: parallel build (make -j <n>) completes without breaking

1.7.0 Release June 2014

  • Support for more than 64k variables in a file
  • File system topology aware I/O method for Titan@OLCF
  • DataSpaces staging
    • support for 64bit dimension sizes
    • support for more than three dimensions
    • works on Bluegene/Q (DataSpaces+DIMES methods)
    • can run as a service (dynamic connections)
  • Additions to non-XML Write API:
    • Support for the visualization schema
    • adios_set_transform() to choose the transformation for a variable
  • Usability improvements:
    • CMake Module for find_package(ADIOS)
    • adios_config -m to print available write/read methods

1.6.0 Release Dec 2013

  • Transformations of data supported in file-based I/O
    • lossless compression (zlib, bzip, szip)
    • lossy compression (ISOBAR)
    • precision-level-of-detail encoding (APLOD)
  • Changes to Write API:
    • variables are identified by full path at writing
    • fix: all int functions return 0 as OK and !=0 on error
  • Changes to Read API:
    • Read API extensions to get information about the visualization meshes defined in a file
    • leading / in path names is not enforced
  • New I/O method for Bluegene/Q called “BGQ” configure with the option –with-bgq
  • Removed performance bottleneck in metadata operations when writing/reading thousands of variables.
  • fix: one can build ADIOS separately from the source with automake

1.5.0 Release June 2013

  • Changes to Write API:
    • adios_init() and adios_init_noxml() has MPI_Comm argument
    • adios_open() has MPI_comm argument instead of void * argument
  • Changes to Read API:
    • adios_read_open_stream() obsolete and now it’s called adios_read_open() indicating that it’s used for both files and staged data.
  • New staging methods:
    • DIMES
  • CMAKE build files (besides Automake files)
  • New write method VAR_MERGE for spatial aggregation of small per-process-output into larger chunks. It improves both write and read performance for such applications.
  • fix: segfault in adios_inq_var_blockinfo()
  • fix: endianness independence ()
  • fix: in adios_inq_var_stat() for getting statistics (avg,
  • fix: backward compatibility in reading old BP files containing scalars over time (a 1D array)

1.4.1 Release Dec 2012

  • aggregated file reader method (from 1.3.1): use ADIOS_READ_METHOD_BP_AGGREGATE
  • memory limitation specified by user for chunked reading now taken into account in file reading
  • stage writer example code for staged I/O see examples/stage_write
  • code coupling example code for file-based/staged coupling see examples/coupling
  • bp2h5 supports converting complex and double complex types
  • new adios_write_byid() function to write multiple-blocks of the same global array from a process
  • fix: F90 modules: adios_write, adios_schedule_read
  • fix: invalid “out of bound” errors when reading multiple steps of multi-dimensional arrays
  • fix: double-free bug in support of old read API in Fortran
  • fix: backward compatibility of old read API to read multi-group files (fixed “invalid variable id…” errors)

1.4.0 Released July 2012

  • new read API for staging methods with step-by-step processing, also with non-blocking and chunking APIs
  • visualization schema added to ADIOS XML format
  • skel: I/O skeleton generator and evaluation tools
  • unified error/debug logging
  • hidden attributes (version, create/update times)
  • Java and Numpy bindings
  • F90 modules adios_write_mod and adios_read_mod to check syntax at compile time

1.3.1 Released Nov 2011

(Download 1.3.1)

  • fix: non-compliant MPI collective calls
  • fix: MPI_AMR method caused MPI_Finalize issues on some clusters
  • fix: histogram calculation: freeing memory too early

1.3 Released July 12, 2011

  • New read method BP_STAGED for reading files with improved performance
  • Changes in MPI_AMR and MPI_LUSTRE to support default parameters. Parameters for MPI_LUSTRE and MPI_AMR in XML file are not mandatory any more.

1.2.1 Released Aug 25, 2010

  • Bug fix in read API (to read arrays written into a file over several timesteps but without specifying time dimension)
  • Added DIMES method from Rutgers for tight-coupling applications

1.2 Released July 15, 2010

  • XML and non-XML write APIs are available
  • More statistics (min/max/avg/ available without overhead in write performance
  • Added MPI_AMR method for advanced mesh refinement codes and for aggregated writing (N procs to P writers to M disks)
  • Added support for subfiles
    • POSIX method many-file output can be read as one file
    • MPI_AMR method uses subfiles to improve write performance
  • Added NetCDF-4 transport method
  • Asynchronous, staging methods on Jaguar XT5 at ORNL
    • DataTap from Georgia Tech
    • DataSpaces from Rutgers
    • NSSI from Sandia
  • MPI_LUSTRE method for best performance on Lustre file system

1.0.1 Released on Dec 9, 2009

  • fix: builds read API on Mac (was broken at utils/bpdump)
  • fix: Fortran written BP files are correctly read by Fortran readers
  • added adios_mpi_stripe2 method for improved performance on Lustre file system

1.0 Released on Nov 18, 2009

Documents & Manuals

User’s Manual & Installation

User’s manual: ADIOS-UsersManual-1.12.0.pdf
Skel manual: skel-doc-1.6.0.pdf
Visualization schema in ADIOS: ADIOS-VisualizationSchema-1.1.pdf
Developer’s manual: ADIOS-DevManual-1.6.0.pdf

Archived Manuals

Press & Publications

This page includes press releases and publications on Adios.


ADIOS Ignites Combustion Simulations

Fusion Gets Faster

Researchers Conduct Breakthrough Fusion Simulation


      1. 1. Liu, Q., Logan, J., Tian, Y., Abbasi, H., Podhorszki, N., Choi, J. Y., Klasky, S., Tchoua, R., Lofstead, J., Oldfield, R., Parashar, M., Samatova, N., Schwan, K., Shoshani, A., Wolf, M., Wu, K. and Yu, W. (2013),

Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks

      1. . Concurrency Computation: Practice and Experience. doi: 10.1002/cpe.3125

      1. 2. Cummings, Klasky, Podhorszki, Barreto, Lofstead, Schwan, Docan, Parashar, Sim, Shoshani, “

EFFIS: and End-to-end Framework for Fusion Integrated Simulation

      1. ”, Proceedings of the 18th Euromicro Conference on Parallel, Distributed and Network-based Processing, PDP 2010, Pisa, Italy, February 17-19, 2010

      1. 3. N. Podhorszki, S. Klasky, Q. Liu, C. Docan, M. Parashar, H. Abbasi, J. Lofstead, K. Schwan, M. Wolf, F. Zheng, J. Cummings. “

Plasma fusion code coupling using scalable I/O services and scientific workflows

      1. “. In Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science (WORKS ’09). ACM, New York, NY, USA, , Article 8 , 9 pages. DOI=10.1145/1645164.1645172

      1. 4. Bent, Gibson, Klasky, Lofstead, Parashar, Polte, Schwan, “

… And eat it too: High read performance in write-optimized HPC I/O middlware file formats

      1. ”. In Proceedings of the 4th Annual Workshop on Petascale Data Storage (2009), pp. 21-25

      1. 5. Abbasi, H., Wolf, M., Eisenhauer, G., Klasky, S., Schwan, K., and Zheng, F. 2009.

DataStager: scalable data staging services for petascale applications

      1. . In Proceedings of the 18th ACM international Symposium on High Performance Distributed Computing (Garching, Germany, June 11 – 13, 2009). HPDC ‘09. ACM, New York, NY, 39-48.

      1. 6. Hasan Abbasi, Jay F. Lofstead, Fang Zheng, Karsten Schwan, Matthew Wolf, Scott Klasky:

Extending I/O through high performance data services

      1. . CLUSTER 2009: 1-10

Version: 1.00