HDF5

From GISAXS
Jump to: navigation, search

Hierarchical Data Format (HDF) is a portable binary format, meant to be extensible and self-describing.

Tools

  • hdfview

Reading

In order to read HDF5 files, you will need a suitable library. Here, we describe how to arrange to open compressed HDF5 files in a Python environment.

Installation: Easy Way

In a debian-based OS (Debian, Ubuntu, etc.), you can install the software from the repository"

  1. sudo add-apt-repository ppa:eugenwintersberger/pni
  2. sudo apt-get update
  3. sudo apt-get install hdf5-plugin-lz4

Installation: Hard Way

These manual installation instructions should work on any Linux system.

  1. Install HDF5 library
    • To install the version from the repository:
      • sudo apt-get install libhdf5-dev
    • In some cases, the repository version may not work (or be out of date). In that case, you can manually download the source from hdfgroup, and install it by doing:
      1. wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.15-patch1.tar.gz
      2. tar xzvf hdf5-1.8.15-patch1.tar.gz
      3. cd hdf5-1.8.15-patch1/
      4. make install
      5. Compilation may take 5-10 minutes. If there are any errors, resolve the dependencies using your package manager.
      6. The library files need to be accessible to other software. This involves adding three of the newly-created folders to the PATH ("hdf5/include/", "hdf5/lib/", "hdf5/bin/"). There are various options:
        • The files can be made globally accessible (to all users of the machine) by moving the header files to "/usr/include/", the libraries to "/usr/lib/" and the binaries to "/usr/bin/" (of course you will need to be root to do so).
        • The files can be made locally accessible by adding their location to the PATH. Edit ~/.bashrc and add (assuming you put your code in ~/local/):
          • export PATH=$PATH:~/local/bin
          • export LD_LIBRARY_PATH=~/local/lib
          • export CPLUS_INCLUDE_PATH=~/local/include
        • Activate the changes by opening a new shell (or running "source ~/.bashrc").
  2. Download the hdf5 plugin from Dectris:
    1. https://github.com/dectris/HDF5Plugin
    2. unzip HDF5Plugin-master.zip
    3. cd HDF5Plugin-master/
    4. make
    5. mkdir ~/local/lib/hdf5plugin
    6. cp lib* ~/local/lib/hdf5plugin/
    7. Obviously, you can select some other location to copy the files in those previous steps. However, the plugin files should be in their own sub-folder, not mixed with other files! One must then set an environment variable for the plugin path. E.g. in "~/.bashrc":
      1. export HDF5_PLUGIN_PATH=~/local/lib/hdf5plugin
  3. If necessary, install dependencies:
    1. Cython (C extensions for Python)
      • sudo apt-get install cython
    2. python-pkgconfig:
  4. Install the h5py package (Python library for HDF5).
    1. Download and extract tarball (e.g. from [1]).
    2. Setup
      • cd h5py
      • sudo python setup.py install
  5. Test the plugin. For example:
import numpy as np
from matplotlib.pyplot import imshow, figure, ion, clim
import h5py

ion()

f = h5py.File("lio3_I_198_data_000001.h5","r")
dset = f['entry']['data']['data']
data = np.array(dset)
f.close()
del dset

img = np.average(data,axis=0)
figure(0)
imshow(img)
clim(0, 5)

See Also