HDF5

Hierarchical Data Format (HDF) is a portable binary format, meant to be extensible and self-describing.

Tools

• hdfview

Reading

In order to read HDF5 files, you will need a suitable library. Here, we describe how to arrange to open compressed HDF5 files in a Python environment.

Installation: Easy Way

In a debian-based OS (Debian, Ubuntu, etc.), you can install the software from the repository"

1. sudo add-apt-repository ppa:eugenwintersberger/pni
2. sudo apt-get update
3. sudo apt-get install hdf5-plugin-lz4

Installation: Hard Way

These manual installation instructions should work on any Linux system.

1. Install HDF5 library
   • To install the version from the repository:
     • sudo apt-get install libhdf5-dev
   • In some cases, the repository version may not work (or be out of date). In that case, you can manually download the source from hdfgroup, and install it by doing:
     1. wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.15-patch1.tar.gz
     2. tar xzvf hdf5-1.8.15-patch1.tar.gz
     3. cd hdf5-1.8.15-patch1/
     4. make install
     5. Compilation may take 5-10 minutes. If there are any errors, resolve the dependencies using your package manager.
     6. The library files need to be accessible to other software. This involves adding three of the newly-created folders to the PATH ("hdf5/include/", "hdf5/lib/", "hdf5/bin/"). There are various options:
        • The files can be made globally accessible (to all users of the machine) by moving the header files to "/usr/include/", the libraries to "/usr/lib/" and the binaries to "/usr/bin/" (of course you will need to be root to do so).
        • The files can be made locally accessible by adding their location to the PATH. Edit ~/.bashrc and add (assuming you put your code in ~/local/):
          • export PATH=$PATH:~/local/bin
          • export LD_LIBRARY_PATH=~/local/lib
          • export CPLUS_INCLUDE_PATH=~/local/include
        • Activate the changes by opening a new shell (or running "source ~/.bashrc").
2. Download the hdf5 plugin from Dectris:
   1. https://github.com/dectris/HDF5Plugin
   2. unzip HDF5Plugin-master.zip
   3. cd HDF5Plugin-master/
   4. make
   5. mkdir ~/local/lib/hdf5plugin
   6. cp lib* ~/local/lib/hdf5plugin/
   7. Obviously, you can select some other location to copy the files in those previous steps. However, the plugin files should be in their own sub-folder, not mixed with other files! One must then set an environment variable for the plugin path. E.g. in "~/.bashrc":
      1. export HDF5_PLUGIN_PATH=~/local/lib/hdf5plugin
3. If necessary, install dependencies:
   1. Cython (C extensions for Python)
      • sudo apt-get install cython
   2. python-pkgconfig:
      • sudo apt-get install python-setuptools
      • wget https://pypi.python.org/packages/source/p/pkgconfig/pkgconfig-1.1.0.tar.gz
      • cd d pkgconfig-1.1.0/
      • python setup.py install
4. Install the h5py package (Python library for HDF5).
   1. Download and extract tarball (e.g. from [1]).
      • wget https://pypi.python.org/packages/source/h/h5py/h5py-2.5.0.tar.gz
      • tar xzvf h5py-2.5.0.tar.gz
   2. Setup
      • cd h5py
      • sudo python setup.py install
5. Test the plugin. For example:

import numpy as np
from matplotlib.pyplot import imshow, figure, ion, clim
import h5py

ion()

f = h5py.File("lio3_I_198_data_000001.h5","r")
dset = f['entry']['data']['data']
data = np.array(dset)
f.close()
del dset

img = np.average(data,axis=0)
figure(0)
imshow(img)
clim(0, 5)

See Also

• Software
• NeXus data format
• Wikipedia:Hierarchical Data Format
• HDF5 Home Page
• HDF5 File Format Specification Version 2.0