Requests with concurrent.futures in Python 2.7

Python 3 is making great steps towrd easy concurrency, and some of those have been backported into python 2.7. The concurrent.futures module is available after you `pip install futures`. This package brings very convinient methods for doing threading (ThreadPool) or multiprocessing (ProcessPool).

Threads are useful when the code is blocked by non bytecode execution, such as I/O or external process execution (C code, system calls, etc). If byte code execution is holding things up, the ProcessPool starts multiple interpreters that can execute in parallel. However, there is more overhead in spinning up these interpreters and in them communicating with the main thread through serialized representations (basically pickle or json over a socket if I understand correctly).

Read More »