titan

Up since 11/8/17 02:45 pm

eos

Up since 11/14/17 11:20 pm

rhea

Up since 10/17/17 05:40 pm

hpss

Up since 11/20/17 09:15 am

atlas1

Up since 11/15/17 07:25 am

atlas2

Up since 11/27/17 10:45 am
OLCF User Assistance Center

Can't find the information you need below? Need advice from a real person? We're here to help.

OLCF support consultants are available to respond to your emails and phone calls from 9:00 a.m. to 5:00 p.m. EST, Monday through Friday, exclusive of holidays. Emails received outside of regular support hours will be addressed the next business day.

h5py

Description

Description

Website: http://code.google.com/p/h5py/

HDF5 for Python (h5py) is a general-purpose Python interface to the Hierarchical Data Format library, version 5. HDF5 is a versatile, mature scientific software library designed for the fast, flexible storage of enormous amounts of data.

From a Python programmer’s perspective, HDF5 provides a robust way to store data, organized by name in a tree-like fashion. You can create datasets (arrays on disk) hundreds of gigabytes in size, and perform random-access I/O on desired sections. Datasets are organized in a filesystem-like hierarchy using containers called “groups”, and accessed using the tradional POSIX /path/to/resource syntax.

Usage

Usage

The following modules are needed to be loaded into the environment:

$ module load python
$ module load python_h5py

Using h5py with python is very straightforward. Import the h5py and numpy python modules. Because of the similarities between h5py and numpy’s datasets, both modules work very well together.

import h5py
import numpy as np

f= h5py.File("testfile.hdf5","w")
dset = f.create_dataset("dset1", (100,), dtype='i')
dset[...] = np.arange(100)

The h5py script is run the same as any other python script. To make sure it worked, you can load the hdf5 module and use h5dump to display the contents of the newly created hdf5 data file.

$ module load cray-hdf5
$ h5dump testfile.hdf5
HDF5 "testfile.hdf5" {
GROUP "/" {
 DATASET "dset1" {
 DATATYPE H5T_STD_I32LE
 DATASPACE SIMPLE { ( 100 ) / ( 100 ) }
 DATA {
 (0): 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
 (19): 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
 (35): 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
 (51): 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66,
 (67): 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82,
 (83): 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98,
 (99): 99
 }
 }
}
}

For a more in depth tutorial, see http://docs.h5py.org/en/latest/quick.html

 

Versions

Available Versions

System Application/Version
Titan python_h5py/2.1.3
Titan python_h5py/2.1.3
Titan python_h5py/2.1.3
Titan python_h5py/2.4.0
Titan python_h5py/2.4.0
Titan python_h5py/2.4.0
Rhea python_h5py/2.5.0