Faster Python Programs through Optimization

Objective

This tutorial provides an overview of techniques to improve the performance of Python programs. The focus is on concepts such as profiling, diffrence of data structures and algorithms as well as a selection of tools an libraries that help to speed up Python.

Intended Audience

Python programmers who would like concepts to improve performance.

Audience Level

Programmers with good Python knowledge.

Prerequisites

Please bring your laptop with the operating system of your choice (Linux, Mac OS X, Windows). In addition to Python 2.6 or 2.7, we need:

• RunSnakeRun
• Guppy_PE framework (<= Python 2.6)
• lineprofiler
• pympler
• mem_profiler
• psyco (Python 2.6 only, version 1.5.2 or higher)
• pypy
• NumPy (version 1.2 or higher).

Method

This is a hands-on course. Students are strongly encouraged to work along with the trainer at the interactive prompt. There will be exercises the students need to do on their own. Experience shows that this active involvement is essential for an effective learning.

Outline

• How fast is fast enough?
• Optimization guidelines
  • Premature optimization
  • Optimization rules
  • Seven steps for incremental optimization
• Optimization strategy
  • Measuring in stones
  • Profiling CPU usage
  • Profiling memory usage
• Algorithms and Anti-patterns
  • String concatenation
  • List and generator comprehensions
  • The right data structure
• Caching
• The example
• Testing speed
• Pure Python
• Meet Psyco, the JIT
• Using PyPy
• NumPy for numeric arrays
• Using multiple CPUs with multiprocessing
• Combination of optimization strategies
• Results of different example implementations

I taught this tutorial multiple times and will update the content as I regularly do.