Automating boring testing activities with tox

My pyelftools project has a comprehensive set of tests, which I make sure to run before every release. This turned out to be a somewhat tiresome manual task, because of the following factors:

1. There are three sets of tests in pyelftools: unit tests, full system tests (comparing the output of my pyelftools-based readelf clone with the real readelf), and example tests (I hate out-of-date examples, so mine are self-testable).
2. I currently promise to keep pyelftools working on Python versions 2.6, 2.7 and 3.2; the tests should be run on all three.
3. I want to test that my setup.py script is alright too, so it would be preferable to build a source distribution, install it in a virtualenv and run the tests there. This also helps me test that the package is correctly pip-installable, and that no artifact of my local setup makes the tests pass by chance.

To a programmer, this list just screams "repetition", and hence some form of automation has to be conjured. Before you rush to roll your own, be sure the check out tox.

tox was designed to solve exactly the problem I described above. Its description is "virtualenv-based automation of test activities" - spot on.

All I have to do in order to automate the testing routine described above is install tox, create a configuration file for it, and make sure to execute tox -rv in the root of my project before every release. Done.

Here's my tox.ini for pyelftools, in all its glory:

[tox]
envlist = py27,py26,py32

[testenv]
commands =
    python test/run_all_unittests.py
    python test/run_examples_test.py
    python test/run_readelf_tests.py

[testenv:py26]
deps =
    unittest2

Basically, tox runs all the commands listed in [testenv] for each "environment" listed in envlist. It creates a virtualenv for each such environment, which is exactly what I wanted.

Moreover, as you can see, tox helps to handle another problem I would have to deal with manually: I'm using the excellent unittest2 package for the pyelftools unit tests. While this package is in the standard library of Python 2.7 and 3.2+, it has to be installed separately in 2.6; tox handles this by providing a deps feature - I can specify the dependencies that need to be installed into the virtualenv for the given environment.