My pyelftools project has a comprehensive set of tests, which I make sure to run before every release. This turned out to be a somewhat tiresome manual task, because of the following factors:

1. There are three sets of tests in pyelftools: unit tests, full system tests (comparing the output of my pyelftools-based readelf clone with the real readelf), and example tests (I hate out-of-date examples, so mine are self-testable).
2. I currently promise to keep pyelftools working on Python versions 2.6, 2.7 and 3.2; the tests should be run on all three.
3. I want to test that my setup.py script is alright too, so it would be preferable to build a source distribution, install it in a virtualenv and run the tests there. This also helps me test that the package is correctly pip-installable, and that no artifact of my local setup makes the tests pass by chance.

To a programmer, this list just screams "repetition", and hence some form of automation has to be conjured. Before you rush to roll your own, be sure the check out tox.

tox was designed to solve exactly the problem I described above. Its description is "virtualenv-based automation of test activities" – spot on.

All I have to do in order to automate the testing routine described above is install tox, create a configuration file for it, and make sure to execute tox -rv in the root of my project before every release. Done.

Here’s my tox.ini for pyelftools, in all its glory:

[tox]
envlist = py27,py26,py32

[testenv]
commands =
    python test/run_all_unittests.py
    python test/run_examples_test.py
    python test/run_readelf_tests.py

[testenv:py26]
deps =
    unittest2

Basically, tox runs all the commands listed in [testenv] for each "environment" listed in envlist. It creates a virtualenv for each such environment, which is exactly what I wanted.

Moreover, as you can see, tox helps to handle another problem I would have to deal with manually: I’m using the excellent unittest2 package for the pyelftools unit tests. While this package is in the standard library of Python 2.7 and 3.2+, it has to be installed separately in 2.6; tox handles this by providing a deps feature – I can specify the dependencies that need to be installed into the virtualenv for the given environment.

Related posts:

1. Python unit testing: parametrized test cases
2. pyelftools ported to Python 3
3. pyelftools – Python library for parsing ELF and DWARF
4. New year’s Python meme 2011
5. unit testing framework – cxxtext

This entry was posted on Monday, May 7th, 2012 at 04:27 and is filed under Python. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.