PyXSanalysis

pyXSanalysis is a series of Python scripts that use extend the pyXS software package. This code is intended to automate various aspects of data-treatment for SAXS and WAXS area images.

Installing (Summary)

1. You will need Python 2.6 installed, including the packages:

   numpy (Numerical Python)

   matplotlib (plotting package)

   PIL (Python Image Library)

   (Optional) GSL (GNU Scientific Library) and PyGSL is necessary for certain advanced features.

2. Simply copy the entire directory of Python scripts somewhere on your machine, and run them.
3. Note that the first time you run the scripts, you may get an error like:

   ImportError: /pyXSanalysis/pyXS/_RQconv.so: wrong ELF class: ELFCLASS32

   To fix this, you will need to use a version of RQconv appropriate to your platform (Windows, Mac, Linux, etc.). Go into the sub-folder "pyXSanalysis/pyXS/RQconv_versions/" and see if there is a directory appropriate for your system. If so, copy all the contents of that directory into "pyXSanalysis/pyXS/". If none of the available RQconv versions work for you, then you will need to compile a new one:

   1. Install SWIG (Simplified Wrapper and Interface Generator)
   2. Go into the pyXSanalysis/pyXS/ sub-directory.
   3. Delete the file "_RQconv.so"
   4. Rebuild a new RQconv by using:

   python setup.py build_ext --inplace

Windows Installation Instructions

1. Copy the Python scripts somewhere appropriate on your computer (e.g. C:\pyXSmain\). The directory should contain "analysis.py" and the sub-directory "pyXSanalysis".
2. Install Python 2.6:
   1. Go to http://www.python.org/download and download the latest version of Python 2.6 (e.g. click Python 2.6.4 Windows installer).
   2. Install python by running the downloaded file (e.g. python-2.6.4.msi)
3. Install necessary Python packages:
   1. numpy (Numerical Python)
      1. Go to Numpy download page and download a recent version (e.g. 1.3.0)
      2. Run the executable (e.g. numpy-1.3.0-win32-superpack-python2.6.exe) and follow the instructions to install.
   2. PIL (Python Image Library)
      1. Go to PIL homepage and download a recent release (e.g. Python Imaging Library 1.1.7 for Python 2.6)
      2. Run the executable (e.g. PIL-1.1.7.win32-py2.6.exe) and follow the instructions to install.
   3. matplotlib (a Python plotting library)
      1. Go to download page and download a recent version (e.g. 0.99.0)
      2. Run the executable (e.g. matplotlib-0.99.1.win32-py2.6.exe) and follow the instructions.
4. Switch to a version of RQconv appropriate to your platform (Windows, Mac, Linux, etc.):
   1. Go into the sub-folder "pyXSanalysis\pyXS\RQconv_versions\"
   2. Find a directory appropriate for your system. The version "x86.Windows7.virtualbox" should work for Windows 7.
   3. Copy the entire contents of the directory into "pyXSanalysis\pyXS\". Thus, you are copying the files "_RQconv.pyd", "RQconv.o", and so on, into the pyXS folder.
   4. If none of the available RQconv versions work for your system, you may get an error like:

      ImportError: /pyXSanalysis/pyXS/_RQconv.so: wrong ELF class: ELFCLASS32

      In that case, you will need to compile a new version of RQconv:

      1. Install MinGW:
         1. From the download page, download the "Automated MinGW Installer" (e.g. MinGW-5.1.6.exe).
         2. Run the executable. When selecting components, include "MinGW base tools", "g++ compiler" and "MinGW Make".
         3. Add appropriate environment variables:
            1. Open the "System Properties" from the Windows Control Panel. On the "Advanced" tab, select "Environment Variables..."
            2. Edit the system variable called "Path" by adding entries at the end (separated by semi-colons):

               C:\python26\;C:\MinGW\bin\

      2. Install SWIG (Simplified Wrapper and Interface Generator) for Python access to C-code:
         1. Go to the download page and download a recent version (e.g. swigwin-1.3.40).
         2. Unzip the package and extract the folder within.
         3. Move the folder (e.g. swigwin-1.3.40) to a sensible location (e.g. C:\swig\).
         4. Add appropriate environment variables:
            1. Open the "System Properties" from the Windows Control Panel. On the "Advanced" tab, select "Environment Variables..."
            2. Edit the system variable called "Path" by adding entries at the end (separated by semi-colons):

               C:\python26\;C:\MinGW\bin\;C:\swig\

            3. Also create two new system variables like:

               PYTHON_INCLUDE: D:\python21\include

               PYTHON_LIB: D:\python21\libs\python21.lib

      3. Build RQConv using SWIG
         1. Open a Windows shell (e.g. Start > Run > "cmd")
         2. Navigate to the proper directory:

            cd C:\pyXSmain\pyXSanalysis\pyXS\

         3. Build the module using the provided "setup.py" script:

            c:\python26\python setup.py build_ext --inplace --compiler=mingw32

      4. Copy the newly-created files (_RQconv.pyd RQconv.py RQconv_wrap.c) into the pyXS directory.
5. Run the Python file:

   python ./analysis.py

How to run on Windows

Once everything is installed, you can right-click on the "analysis.py" file, and select "Edit with IDLE...". This will open the file. You can then select "Run" (F5) to run the code, and edit the code in "analysis.py" in order to do different kinds of analyses. You can browse through the file "pyXSanalysis\processing.py" to see what kinds of analysis routines are available (and what options they take). You can look in "examples\various.py" to see some examples of code you could put into "analysis.py".

Troubleshooting the RQconv compiling

1. If you get an error message like:

   C:\pyXSmain\pyXSanalysis\pyXS>python setup.py build_ext --inplace

   running build_ext

   building '_RQconv' extension

   swigging RQconv.i to RQconv_wrap.c

   C:\swig\swig.exe -python -o RQconv_wrap.c RQconv.i

   RQconv.i(9): Error: Unable to find 'numpy.i'

   error: command 'swig.exe' failed with exit status 1

   Then you need to find the "numpy.i" interface file and let swig find it. You can browse the numpy code and download numpy.i. Then place it in the build directory (C:\pyXSmain\pyXSanalysis\pyXS\).

2. If you get an error message like:

   C:\pyXSmain\pyXSanalysis\pyXS>python setup.py build_ext --inplace

   running build_ext

   building '_RQconv' extension

   swigging RQconv.i to RQconv_wrap.c

   C:\swig\swig.exe -python -o RQconv_wrap.c RQconv.i

   error: Unable to find vcvarsall.bat

   Then you probably need to install MinGW correctly and specify it as the compiler you wish to use (--compiler=mingw32 flag).

Mac OS X Installation Instructions

1. Copy the Python scripts somewhere appropriate on your computer (e.g. ~/Desktop/pyXSmain/). The directory should contain "analysis.py" and the sub-directory "pyXSanalysis".
2. Python should already be installed. (If not, go here.)
3. Install necessary Python packages:
   1. PIL (Python Image Library)
      1. Go to PIL homepage and download a recent release (e.g. Python Imaging Library 1.1.7 Source Kit)
      2. Extract the tar file (double-click on file, or in terminal use: tar xzvf Imaging-1.1.7.tar.gz
      3. Go into the folder that has just been created (cd Imaging-1.1.7)
      4. Build:

         python setup.py install

Troubleshooting

Python Image Library (PIL)

1. If you get the error:

   lipo: can't open input file: /var/tmp//ccG8BgoP.out (No such file or directory)

   error: command 'gcc-4.2' failed with exit status 1

   Then you need to manually re-run the gcc command that failed, but remove the unnecessary "-arch PPC" flag:

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c _imaging.c -o build/temp.macosx-10.6-universal-2.6/_imaging.o

   And keep building, removing "-arch ppc" as needed:

   python setup.py build

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c decode.c -o build/temp.macosx-10.6-universal-2.6/decode.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c encode.c -o build/temp.macosx-10.6-universal-2.6/encode.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c map.c -o build/temp.macosx-10.6-universal-2.6/map.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c display.c -o build/temp.macosx-10.6-universal-2.6/display.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c outline.c -o build/temp.macosx-10.6-universal-2.6/outline.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c path.c -o build/temp.macosx-10.6-universal-2.6/path.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Access.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Access.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Antialias.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Antialias.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Bands.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Bands.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/BitDecode.c -o build/temp.macosx-10.6-universal-2.6/libImaging/BitDecode.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/BitDecode.c -o build/temp.macosx-10.6-universal-2.6/libImaging/BitDecode.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Blend.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Blend.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Chops.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Chops.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Convert.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Convert.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/ConvertYCbCr.c -o build/temp.macosx-10.6-universal-2.6/libImaging/ConvertYCbCr.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Copy.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Copy.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Crc32.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Crc32.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Crop.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Crop.o

   gcc -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/sw/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c libImaging/Dib.c -o build/temp.macosx-10.6-universal-2.6/libImaging/Dib.o

   Etc.