Decorators and Context managers§

Twitter: @andreacrotti

_images/wazoku.png

https://github.com/AndreaCrotti/ep2013

Background§

def func(arg1, arg2):
    var1 = 10
>>> type(func)
>>> function
class X(object):
    pass
>>> type(X)
>>> type

Decorator§

In python a decorator is a callable that takes a function (or a class) as argument, and returns a function (or a class) with an altered behaviour.

The @ decorator syntax for functions has been introduced in Python 2.4, and for classes in Python 2.6 (see decorator-history).

Shocking example§

def fib(n):
    if n <= 1:
        return 1
    return fib(n - 1) + fib(n - 2)
@memoize
def fib_memoized(n):
    if n <= 1:
        return 1
    return fib_memoized(n - 1) + fib_memoized(n - 2)
n fib fib_memoized speedup
5 2.66 μs 1.16 μs 2x
20 3780 μs 1.21 μs 3000x

Hello decorator§

def decorator(func):
    def _decorator(*args, **kwargs):
        # something before the function is run
        ret = func(*args, **kwargs)
        # something after the function is run
        return ret

    return _decorator
@decorator
def my_function(): pass

Which is simply syntactic sugar for:

def my_function(): pass
my_function = decorator(my_function)

Why the _decorator?§

def naive_decorator(func, *args, **kwargs):
    # pre actions

    func(*args, **kwargs)

    # post actions
@naive_decorator
def my_function(): pass
my_function = naive_decorator(my_function)

Here the function get immediately executed!, returning None

Simple example§

def verbose(original_function):
    def new_function(*args, **kwargs):
        print("Entering function %s" % original_function.func_name)
        ret = original_function(*args, **kwargs)
        print("Exiting function %s" % original_function.func_name)
        return ret

    return new_function
>>> def silly_func():
>>>     print("Simple function")

>>> silly_func = verbose(silly_func)
Entering function silly_func
Simple function
Exiting function silly_func

Back to memoization§

memoize caches the results of generic function calls.

def memoize(func, cache={}):
    def _memoize(*args, **kwargs):
        # create an hashable key for the cache dict
        key = (args, str(kwargs))
        if not key in cache:
            cache[key] = func(*args, **kwargs)
        else:
            print("Cache hit for key = %s" % str(key))

        return cache[key]
    return _memoize

Completely generic, memoize any recursive function

See lru_cache for a safer implementation: http://hg.python.org/cpython/file/8e838598eed1/Lib/functools.py

Memoization unfolded§

step# call cache
1 fib(4) {}
2 (fib(3) + fib(2)) {}
3 ((fib(2) + fib(1)) + fib(2)) {}
4 (((fib(1) + fib(0)) + fib(1)) + fib(2)) {}
5 1 + 1 + fib(1) + fib(2) {fib((2, )): 2}
6 2 + 1 + 2 5

Parametric decorator 1§

How do I parametrize a decorator?

def param_deco(doit=False):
    def _param_deco(f):
        def __param_deco(*args, **kwargs):
            if doit:
                print("Running function")
                return f(*args, **kwargs)
            else:
                print("f on input %s, %s" % (str(args), str(kwargs)))

        return __param_deco
    return _param_deco

Parametric decorator 2§

Or alternatively overriding the __call__ method.

class param_deco_class(object):
    def __init__(self, doit=False):
        self.doit = doit

    def __call__(self, func):
        def _wrap(*args, **kwargs):
            if self.doit:
                print("Running function")
                return f(*args, **kwargs)
            else:
                print("f on input %s, %s" % (str(args), str(kwargs)))

        return _wrap
    

Parametric decorator 3§

class retry_n_times(object):
    def __init__(self, ntimes=3, timeout=3):
        self.ntimes = ntimes
        self.timeout = timeout

    def __call__(self, func):
        def _retry_n_times(*args, **kwargs):
            attempts = 0
            while attempts < self.ntimes:
                try:
                    ret = func(*args, **kwargs)
                except Exception:
                    sleep(self.timeout)
                else:
                    return ret
                attempts += 1

        return _retry_n_times

Class decorator§

def class_decorator(cls):
    def new_meth(self):
        return 100

    cls.new_meth = new_meth
    return cls
@class_decorator
class C1(object):
    pass
>>> C1().new_meth()
>>> 100

Patching classes§

import unittest

from mock import patch

@patch('ext.module.function', new=lambda x: 42)
class TestMyClass(unittest.TestCase):
    def setUp(self):
        pass
    # ...

Which applies the patch for all the methods found by inspection.

Context manager§

Introduced in Python 2.5 with the with_statement.

A context manager is useful whenever you can split the actions in:

With statement§

The idea is to not forget cleanup actions.

with open('file.txt') as source:
     text = source.read()
     # a lot
     # more
     # code

Is equivalent to:

source = open('file.txt')
text = source.read()
# a lot
# more
# code
source.close()

Implement __enter__ and __exit__§

class TempFile:
    """Create a temporary file with the given content and remove it on exit
    """
    def __init__(self, content=None, temp_path=None):
        self.content = content or ""
        self.temp_file = mktemp() if temp_path is None else temp_path

    def __enter__(self):
        with open(self.temp_file, 'w') as wr:
            wr.write(self.content)

        return self.temp_file

    def __exit__(self, type, value, traceback):
        remove(self.temp_file)

Add that there can be an exception handling in the with.

Using contextlib§

Contextmanager runs the generator until yield, then stops and runs until the end.

from contextlib import contextmanager

@contextmanager
def tag(name):
    print("<%s>" % name)
    yield
    print("</%s>" % name)
>>> with tag('H1'):
>>>      print('Title')

'<H1>Title</H1>'

Thanks§

_images/questions.jpg

@andreacrotti https://github.com/AndreaCrotti/ep2013