Python:Speed

Python is a powerful high-level programming language with a clean syntax. However, the flexibility and generality (e.g. dynamic typing) does have an associated performance cost. There are various strategies to improve the speed of execution of Python code:

1. Re-code: As always, the first thing to try is to identify the bottleneck in the code, and rework it. Typically, using the most appropriate algorithm can improve execution by orders-of-magnitude.
2. Libraries: Exploiting Python libraries (which are highly optimized and often written in lower-level languages) can greatly improve performance. In particular, using numpy for matrix-style numerical computations (rather than using expensive for-loops or other iterations) can massively speedup computations. Processing on images can be improved using the Python Image Library (PIL), etc.
3. Externals: Critical code can be written in C/C++, and called as a function within Python. This allows the computational bottleneck to be written in a more specialized and efficient manner.
   • SWIG (official site, Wikipedia) can provide a 50-200× speedup.
4. JIT: Just-in-time compilation (JIT) involves compiling Python code as it is needed. The compiling adds a speed penalty as code is first run, but improves overall execution speed if the code iterates over a large dataset.
   • Psycho (official site, Wikipedia) provides a 2-4× speedup (100× in some cases). It is only 32-bit.
   • PyPy (official site, Wikipedia) is an alternative Python interpretation, which features JIT. It provides a 2-25× speedup. Unfortunately, modules/libraries have to be re-installed/re-compiled into the PyPy environment (separate from the usual Python environment).
5. Translation: There are some attempts to automatically translate Python code into optimized lower-level code.
   • shedskin (official site 1, official site 2, Wikipedia) translates Python into C++, providing a 2-200× speedup. Most extensions/libraries are not currently supported. On the other hand, one can isolate some critical code and convert this to an optimized external that is called from conventional Python code.