titan

Up since 11/8/17 02:45 pm

eos

Up since 11/14/17 11:20 pm

rhea

Up since 10/17/17 05:40 pm

hpss

Up since 11/20/17 09:15 am

atlas1

Up since 11/15/17 07:25 am

atlas2

Up since 11/27/17 10:45 am
OLCF User Assistance Center

Can't find the information you need below? Need advice from a real person? We're here to help.

OLCF support consultants are available to respond to your emails and phone calls from 9:00 a.m. to 5:00 p.m. EST, Monday through Friday, exclusive of holidays. Emails received outside of regular support hours will be addressed the next business day.

adios

Description

Description

Website: http://www.olcf.ornl.gov/center-projects/adios/

The Adaptable IO System (ADIOS) provides a simple, flexible way for scientists to describe the data in their code that may need to be written, read, or processed outside of the running simulation. By providing an external to the code XML file describing the various elements, their types, and how you wish to process them this run, the routines in the host code (either Fortran or C) can transparently change how they process the data.

ADIOS has demonstrated impressive IO performance results on leadership class machines and clusters; sometimes showing an improvement of more than 1000 times over well known parallel file formats. ADIOS is essentially an I/O componentization of different I/O transport methods. This feature allows flexibility for application scientists to adopt the best I/O method for different computer infrastructures with very little modification of their scientific applications. ADIOS has a suite of simple, easy-to-use APIs. Instead of being provided as the arguments of APIs, all the required metadata are stored in an external Extensible Markup Language (XML) configuration file, which is readable, editable, and portable for most machines.

Usage

Usage

ADIOS is available as a module.

$ module load adios/1.9.0

There are several versions of ADIOS. The default version of the module includes POSIX, MPI-IO and MPI-CIO methods. We also have parallel HDF5 as seperate transport method.

For parallel HDF5:

module load hdf5/1.8.1
ftn -c test.F90 ${ADIOS_INC}
ftn -o test test.o ${ADIOS_LIB} ${HDF5_FLIB}

or

module load hdf5/1.8.1
CC -c test.F90 ${ADIOS_INC}
CC -o test test.o ${ADIOS_LIB} ${HDF5_CLIB}

Examples can be found in the examples directory of ADIOS. A simple example code would be:

arrays_write.c:

#include <stdio.h>
#include <string.h>
#include "mpi.h"
#include "adios.h"

/*************************************************************/
/*          Example of writing arrays in ADIOS               */
/*                                                           */
/*        Similar example is manual/2_adios_write.c          */
/*************************************************************/
int main (int argc, char ** argv) 
{
    char        filename [256];
    int         rank, i, j;
    int         NX = 10, NY = 100; 
    double      t[NX][NY];
    int         p[NX];
    MPI_Comm    comm = MPI_COMM_WORLD;

    uint64_t    adios_groupsize, adios_totalsize;
    int64_t     adios_handle;

    MPI_Init (&argc, &argv);
    MPI_Comm_rank (comm, &rank);

    for (i = 0; i < NX; i++)
        for (j = 0; j< NY; j++)
            t[i][j] = rank * NX + i + j*(1.0/NY);

    for (i = 0; i < NX; i++)
        p[i] = rank * NX + i;

    strcpy (filename, "arrays.bp");
    adios_init ("arrays.xml", comm);
    adios_open (&adios_handle, "arrays", filename, "w", comm);
#include "gwrite_arrays.ch"
    adios_close (adios_handle);

    MPI_Barrier (comm);

    adios_finalize (rank);

    MPI_Finalize ();
    return 0;
}

gwrite_arrays.ch:

adios_groupsize = 4 \
                + 4 \
                + 8 * (NX) * (NY) \
                + 4 * (NX);
adios_group_size (adios_handle, adios_groupsize, &adios_totalsize);
adios_write (adios_handle, "NX", &NX);
adios_write (adios_handle, "NY", &NY);
adios_write (adios_handle, "var_double_2Darray", t);
adios_write (adios_handle, "var_int_1Darray", p);

arrays.xml:

<?xml version="1.0"?>
<adios-config host-language="C">

 <adios-group name="arrays">
 <var name="NX" type="integer"/>
 <var name="NY" type="integer"/>
 <var name="var_double_2Darray" gwrite="t" type="double" dimensions="NX,NY"/>
 <var name="var_int_1Darray" gwrite="p" type="integer" dimensions="NX"/>
 </adios-group>

 <method group="arrays" method="MPI"/>

 <buffer max-size-MB="20"/>

</adios-config>

The xml file is used to provide ADIOS information on how to format the data. Compilation is the same as above. Make sure that the XML file is in the same directory that the binary is aprun’d from!

Versions

Available Versions

System Application/Version
Titan adios/1.4.1pre1
Titan adios/1.6.0
Titan adios/1.7.0
Titan adios/1.8.0
Titan adios/1.9.0
Rhea adios/1.4.1
Rhea adios/1.5.0
Rhea adios/1.6.0
Rhea adios/1.7.0
Rhea adios/1.8.0
Rhea adios/1.9.0(default)
Eos adios/1.5.0
Eos adios/1.6.0
Eos adios/1.8.0
Eos adios/1.8.0_sles11