Requests with concurrent.futures in Python 2.7

Python 3 is making great steps towrd easy concurrency, and some of those have been backported into python 2.7. The concurrent.futures module is available after you `pip install futures`. This package brings very convinient methods for doing threading (ThreadPool) or multiprocessing (ProcessPool).

Threads are useful when the code is blocked by non bytecode execution, such as I/O or external process execution (C code, system calls, etc). If byte code execution is holding things up, the ProcessPool starts multiple interpreters that can execute in parallel. However, there is more overhead in spinning up these interpreters and in them communicating with the main thread through serialized representations (basically pickle or json over a socket if I understand correctly).

Read More »

An Agile Time Keeping App

I saw the comedy theater I intern at could really benefit from a computerized way of tracking when interns show up. So, I decided to practice my agile programming and whip something up as quickly as possible. I considered using a web framework, but decided that Django would be overkill for such a simple project.

So, where to start? I need to keep track of users, who have passwords. I decided a dictionary was the way to go.
Read More »

Simple web scraper

From time to time there are situations where it is helpful to quickly rip some information from a group of web pages.  For example, I was doing some keyword analysis and wanted to know how many Google results there are for each keyword in my list (both exact and broad matches).  The answer is right there on the page when you do the Google search, so all there is too it is to automate the process of doing queries.
Read More »