Floris Bruynooghe

email: flub@devork.be

twitter: @flubdevork

irc: flub

• Echo client/server - blocking, select, theading
• Greenlets
• Eventlet
• (Monkey)patching
• Multiple threads

• Concurrent networking library
• Coroutines for blocking-style code
• Implicit dispatch

Me:

• Using eventlet since ~2010
• Retrofitted into multi-threaded application

sock = socket.socket()
sock.connect(('localhost', 7))
sock.send('Hello world')  # <- blocks
sock.recv(4096)           # <- blocks
sock.close()

• Server response: "hello world"
• No harm from blocking

srv_sock = socket.socket()
srv_sock.bind(('', 7))
srv_sock.listen(5)
while True:
    client, addr = srv_sock.accept()  # <- blocks
    data = client.recv(4096)          # <- blocks
    while data:
        client.send(data)             # <- blocks
        data = client.recv(4096)      # <- blocks
    client.close()

• WYSIWYG
• Single client at a time
• Not how you write a server (IPv6, hardcoded numbers, ...)

srv_sock = socket.socket()
srv_sock.bind(('', 7))
srv_sock.listen(5)
while True:
    client, addr = srv_sock.accept()
    t = threading.Thread(target=handle_client, args=client)
    t.start()

def handle_client(sock):
    data = sock.recv(4096)
    while data:
        sock.send(data)
        data = sock.recv(4096)
    sock.close()

• Many clients in parallel
• OSes are good at this!
• #threads < #fds
• Context switching can be heavy

srv_sock = socket.socket()
srv_sock.bind(('', 7))
srv_sock.listen(5)
rlist = [srv_sock]
wlist = clients = []
while True:
    readers, writers, _ = select.select(rlist, wlist, [])
    for client in clients:
        if client in readers and client in writers:
            data = client.recv(4096)
            if data == b'':
                clients.remove(client)
            else:
                client.send(data)
    if srv_sock in readers:
        clients.append(srv_sock.accept()[0])
    rlist = [srv_sock] + clients

• wlist is clients
• Big, ugly, need to know a lot more
• Even buggier then blocking version

srv_sock = eventlet.green.socket.socket()
srv_sock.bind(('', 7))
srv_sock.listen(5)
while True:
    client, addr = srv_sock.accept()          # <- yield
    eventlet.spawn_n(handle_client, client)

def handle_client(sock):
    data = sock.recv(4096)  # <- yield
    while data:
        sock.send(data)     # <- yield
        data = sock.re
    sock.close()

• No starting of greenlet: hub will run it

• Userland "context switches"
  • Multiple "green threads" in one real thread
• Created by Christian Tismer and Armin Rigo
  • Maintained by Ralph Smith (and Alexey Borzenkov).
• Grew out of stackless Python
• Extension module for CPython
• Built-in into pypy (+ JITed!)
• Very cumbersome
• Base for concurrence, eventlet, gevent

main = greenlet.getcurrent()
l = []
def func(other):
    l.append(0)
    other.switch()
    l.append(2)
g = greenlet.greenlet(run=func)
g.switch(main)
l.append(1)
g.switch()
assert l == [0, 1, 2]
assert g.dead

• Suspend/resume execution
• Copies C stack onto the heap

• Mainloop running in a greenlet
• Blocking == switch to hub implicitly
• Uses "best" I/O selection method
  • select
  • poll
  • epoll
  • kqueue
  • ...
• Spawn new greenlets using eventlet.spawn()
  • Schedules new greenlet with hub

• Does not switch
• Allows retrieval of result
• Waiting on completion
• spawn_n() does not return greenlet object

def fun():
    eventlet.sleep(5)
    return 42

g = eventlet.spawn(fun)
assert bool(g) is False
eventlet.sleep(0)               # yield
assert bool(g) is True
assert g.dead is False
rv = g.wait()
assert rv == 42

Concurrency, but not too much

srv_sock = eventlet.green.socket.socket()
srv_sock.bind(('', 7))
srv_sock.listen(5)
pool = eventlet.GreenPool(1000)
while True:
    client = srv_sock.accept()
    pool.spawn_n(handle_client, client)

def handle_client(sock):
    ...

eventlet.monkey_patch()

srv_sock = socket.socket()
srv_sock.bind(('', 7))
srv_sock.listen(5)
while True:
    client = srv_sock.accept()
    t = threading.Thread(target=handle_client, args=client)
    t.start()

def handle_client(sock):
    data = sock.recv(4096)
    while data:
        sock.send(data)
        data = sock.recv(4096)
    sock.close()

• Bad egineering:
  • Location/import order matters
  • Hidden effect on entire application; Abstractions always leak
• No longer access to real threads

• Explicit which modules cooperate with greenlets
• Can mix non-greened modules
• from eventlet.green import socket
• requests = eventlet.import_patched('requests')
  • Beware, current patching of packages patchy
  • Do not used unpatched anymore
  • Will be fixed
• Pure python only

• Threadpool for blocking calls
• Only blocking/C call, no more
  • Don't start hub in a thread
  • Can be tricky with ORMs
  • No logging calls!
  • Will be fixed

ai = eventlet.tpool.execute(
    socket.getaddrinfo, 'www.kame.net', None, socket.AF_INET6)

• See https://bitbucket.org/flub/eventlet-ipv6 for eventlet IPv6 support (incomplete).

Real threads:

• Parallel (sometimes: remember the GIL)
• Scheduled: each thead gets a time slice
• Signalling, locking, ...

Green threads:

• Concurrent
• Explicit switching: don't do lots of CPU
• No full context-switch: fast

Using multiple threads:

• Some running eventlet's hub
• Some without eventlet's hub

• Queue.Queue -> threads only
• eventlet.green.Queue.Queue -> greenlets only
• eventlet.xthread.Queue - https://bitbucket.org/flub/eventlet-queue/
• 0mq?

q = eventlet.xthread.Queue()

def producer():
    q.put(42)

def consumer():
    return p.get()

t = threading.Thread(target=producer)
g = eventlet.spawn(consumer)
t.start()
assert g.wait() == 42

Blocking:

• Greenlet switches to hub
• Can never acquire a real lock
  • Would inhibit other greenlets from running
  • Atomic operations only

Unblocking:

• Switch back to the greenlet
• Same hub: direct switch
• Other thread:
  • Schdule switch with hub
  • Wake up remote hub (pipe)

Blocking:

• Acquire a locked lock

Unblocking:

• Release the lock
  • Safe for a greenlet

• Always know which environment you are in
  • Don't accidentically start a hub
• Default implementations not cross-thread aware

(Q4 2013)

• Xthread likely to be merged
• Default implementations to use xthread
  • Safe to use python code in tpool
• Python 3
• IPv6 (address resolution)

In Summary:

• Use eventlet
• Don't monkeypatch
• Dare to design using multiple threads

Questions?