Difference between revisions of "HDF5"

From GISAXS
Jump to: navigation, search
(→‎Installation)
(→‎Reading)
 
(7 intermediate revisions by the same user not shown)
Line 5: Line 5:
  
 
==Reading==
 
==Reading==
−
In order to read HDF5 files, you will need a suitable library.
+
In order to read HDF5 files, you will need a suitable library. Here, we describe how to arrange to open compressed HDF5 files in a [[Python]] environment.
−
===Installation===
+
===Installation: Easy Way===
−
# Install dependencies
+
In a debian-based OS (Debian, Ubuntu, etc.), you can install the software from the repository"
−
#* sudo apt-get install libhdf5-dev
+
# sudo add-apt-repository ppa:eugenwintersberger/pni
−
#* sudo apt-get install cython
+
# sudo apt-get update
−
# If necessary, install [https://pypi.python.org/pypi/pkgconfig/1.1.0 python-pkgconfig]:
+
# sudo apt-get install hdf5-plugin-lz4
−
#* sudo apt-get install python-setuptools
+
 
−
#* wget https://pypi.python.org/packages/source/p/pkgconfig/pkgconfig-1.1.0.tar.gz
+
===Installation: Hard Way===
−
#* cd d pkgconfig-1.1.0/
+
These manual installation instructions should work on any Linux system.
−
#* python setup.py install
+
# Install HDF5 library
 +
#* To install the version from the repository:
 +
#** sudo apt-get install libhdf5-dev
 +
#* In some cases, the repository version may not work (or be out of date). In that case, you can manually download the source from [https://www.hdfgroup.org/HDF5/release/obtainsrc.html#src hdfgroup], and install it by doing:
 +
#*# wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.15-patch1.tar.gz
 +
#*# tar xzvf hdf5-1.8.15-patch1.tar.gz
 +
#*# cd hdf5-1.8.15-patch1/
 +
#*# make install
 +
#*# Compilation may take 5-10 minutes. If there are any errors, resolve the dependencies using your package manager.
 +
#*# The library files need to be accessible to other software. This involves adding three of the newly-created folders to the PATH ("hdf5/include/", "hdf5/lib/", "hdf5/bin/"). There are various options:
 +
#*#* The files can be made globally accessible (to all users of the machine) by moving the header files to "/usr/include/", the libraries to "/usr/lib/" and the binaries to "/usr/bin/" (of course you will need to be root to do so).
 +
#*#* The files can be made locally accessible by adding their location to the PATH. Edit ~/.bashrc and add (assuming you put your code in ~/local/):
 +
#*#** export PATH=$PATH:~/local/bin
 +
#*#** export LD_LIBRARY_PATH=~/local/lib
 +
#*#** export CPLUS_INCLUDE_PATH=~/local/include
 +
#*#* Activate the changes by opening a new shell (or running "source ~/.bashrc").
 +
# Download the hdf5 plugin from [https://www.dectris.com/ Dectris]:
 +
## [https://github.com/dectris/HDF5Plugin https://github.com/dectris/HDF5Plugin]
 +
## unzip HDF5Plugin-master.zip
 +
## cd HDF5Plugin-master/
 +
## make
 +
## mkdir ~/local/lib/hdf5plugin
 +
## cp lib* ~/local/lib/hdf5plugin/
 +
## Obviously, you can select some other location to copy the files in those previous steps. However, the plugin files should be in their ''own sub-folder'', not mixed with other files! One must then set an environment variable for the plugin path. E.g. in "~/.bashrc":
 +
### export HDF5_PLUGIN_PATH=~/local/lib/hdf5plugin
 +
# If necessary, install dependencies:
 +
## [http://cython.org/ Cython] (C extensions for Python)
 +
##* sudo apt-get install cython
 +
## [https://pypi.python.org/pypi/pkgconfig/1.1.0 python-pkgconfig]:
 +
##* sudo apt-get install python-setuptools
 +
##* wget https://pypi.python.org/packages/source/p/pkgconfig/pkgconfig-1.1.0.tar.gz
 +
##* cd d pkgconfig-1.1.0/
 +
##* python setup.py install
 
# Install the [http://www.h5py.org/ h5py package] ([[Python]] library for HDF5).
 
# Install the [http://www.h5py.org/ h5py package] ([[Python]] library for HDF5).
 
## Download and extract tarball (e.g. from [https://pypi.python.org/pypi/h5py]).
 
## Download and extract tarball (e.g. from [https://pypi.python.org/pypi/h5py]).
Line 22: Line 54:
 
##* cd h5py
 
##* cd h5py
 
##* sudo python setup.py install
 
##* sudo python setup.py install
 +
# Test the plugin. For example:
 +
 +
<source lang="python" line >
 +
import numpy as np
 +
from matplotlib.pyplot import imshow, figure, ion, clim
 +
import h5py
 +
 +
ion()
 +
 +
f = h5py.File("lio3_I_198_data_000001.h5","r")
 +
dset = f['entry']['data']['data']
 +
data = np.array(dset)
 +
f.close()
 +
del dset
 +
 +
img = np.average(data,axis=0)
 +
figure(0)
 +
imshow(img)
 +
clim(0, 5)
 +
</source>
  
 
==See Also==
 
==See Also==

Latest revision as of 13:56, 20 October 2015

Hierarchical Data Format (HDF) is a portable binary format, meant to be extensible and self-describing.

Tools

  • hdfview

Reading

In order to read HDF5 files, you will need a suitable library. Here, we describe how to arrange to open compressed HDF5 files in a Python environment.

Installation: Easy Way

In a debian-based OS (Debian, Ubuntu, etc.), you can install the software from the repository"

  1. sudo add-apt-repository ppa:eugenwintersberger/pni
  2. sudo apt-get update
  3. sudo apt-get install hdf5-plugin-lz4

Installation: Hard Way

These manual installation instructions should work on any Linux system.

  1. Install HDF5 library
    • To install the version from the repository:
      • sudo apt-get install libhdf5-dev
    • In some cases, the repository version may not work (or be out of date). In that case, you can manually download the source from hdfgroup, and install it by doing:
      1. wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.15-patch1.tar.gz
      2. tar xzvf hdf5-1.8.15-patch1.tar.gz
      3. cd hdf5-1.8.15-patch1/
      4. make install
      5. Compilation may take 5-10 minutes. If there are any errors, resolve the dependencies using your package manager.
      6. The library files need to be accessible to other software. This involves adding three of the newly-created folders to the PATH ("hdf5/include/", "hdf5/lib/", "hdf5/bin/"). There are various options:
        • The files can be made globally accessible (to all users of the machine) by moving the header files to "/usr/include/", the libraries to "/usr/lib/" and the binaries to "/usr/bin/" (of course you will need to be root to do so).
        • The files can be made locally accessible by adding their location to the PATH. Edit ~/.bashrc and add (assuming you put your code in ~/local/):
          • export PATH=$PATH:~/local/bin
          • export LD_LIBRARY_PATH=~/local/lib
          • export CPLUS_INCLUDE_PATH=~/local/include
        • Activate the changes by opening a new shell (or running "source ~/.bashrc").
  2. Download the hdf5 plugin from Dectris:
    1. https://github.com/dectris/HDF5Plugin
    2. unzip HDF5Plugin-master.zip
    3. cd HDF5Plugin-master/
    4. make
    5. mkdir ~/local/lib/hdf5plugin
    6. cp lib* ~/local/lib/hdf5plugin/
    7. Obviously, you can select some other location to copy the files in those previous steps. However, the plugin files should be in their own sub-folder, not mixed with other files! One must then set an environment variable for the plugin path. E.g. in "~/.bashrc":
      1. export HDF5_PLUGIN_PATH=~/local/lib/hdf5plugin
  3. If necessary, install dependencies:
    1. Cython (C extensions for Python)
      • sudo apt-get install cython
    2. python-pkgconfig:
  4. Install the h5py package (Python library for HDF5).
    1. Download and extract tarball (e.g. from [1]).
    2. Setup
      • cd h5py
      • sudo python setup.py install
  5. Test the plugin. For example:
import numpy as np
from matplotlib.pyplot import imshow, figure, ion, clim
import h5py

ion()

f = h5py.File("lio3_I_198_data_000001.h5","r")
dset = f['entry']['data']['data']
data = np.array(dset)
f.close()
del dset

img = np.average(data,axis=0)
figure(0)
imshow(img)
clim(0, 5)

See Also

Retrieved from "http://gisaxs.com/index.php?title=HDF5&oldid=5206"