Python unit testing: parametrized test cases

Python's standard unittest library is great and I use it all the time. One thing missing from it, however, is a simple way of running parametrized test cases. In other words, you can't easily pass arguments into a unittest.TestCase from outside.

Consider the use case: I have some TestCase I want to invoke several times, each time passing it a different argument.

One approach often mentioned is create a base TestCase for the bulk functionality and derive sub-classes from it for variations. But this isn't flexible enough - what if you want to add parameters from the outside (command-line) or test with a large amount of parameters?

Fortunately, Python is dynamic enough (and unittest flexible enough) to allow a relatively straightforward solution.

Here's a class that makes it possible:

import unittest

class ParametrizedTestCase(unittest.TestCase):
    """ TestCase classes that want to be parametrized should
        inherit from this class.
    """
    def __init__(self, methodName='runTest', param=None):
        super(ParametrizedTestCase, self).__init__(methodName)
        self.param = param

    @staticmethod
    def parametrize(testcase_klass, param=None):
        """ Create a suite containing all tests taken from the given
            subclass, passing them the parameter 'param'.
        """
        testloader = unittest.TestLoader()
        testnames = testloader.getTestCaseNames(testcase_klass)
        suite = unittest.TestSuite()
        for name in testnames:
            suite.addTest(testcase_klass(name, param=param))
        return suite

Before I explain how this works, here's a sample usage. Let's define some test case that can be parametrized with an extra param argument:

class TestOne(ParametrizedTestCase):
    def test_something(self):
        print 'param =', self.param
        self.assertEqual(1, 1)

    def test_something_else(self):
        self.assertEqual(2, 2)

Note how nothing except inheriting ParametrizedTestCase is required. self.param automagically becomes available in all test methods (as well as in setUp, tearDown, etc.)

And here is how to create and run parametrized instances of this test case:

suite = unittest.TestSuite()
suite.addTest(ParametrizedTestCase.parametrize(TestOne, param=42))
suite.addTest(ParametrizedTestCase.parametrize(TestOne, param=13))
unittest.TextTestRunner(verbosity=2).run(suite)

As expected, we get:

test_something (__main__.TestOne) ... param = 42
ok
test_something_else (__main__.TestOne) ... ok
test_something (__main__.TestOne) ... param = 13
ok
test_something_else (__main__.TestOne) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

Now, a word on how ParametrizedTestCase works. It's a subclass of unittest.TestCase, and the parametrization is done by defining its own constructor, which is similar to TestCase's constructor but adds an extra param argument. This param is then saved as the instance attribute self.param. Test cases interested in being parametrized should then derive from ParametrizedTestCase.

To actually create the parametrized test, ParametrizedTestCase.parametrize should be invoked. It accepts two arguments:

1. A subclass of ParametrizedTestCase - essentially our custom test case class
2. The parameter we want to pass to this instance of the test case

And then uses the test name discovery facilities available in unittest.TestLoader to create the tests and parametrize them.

As you can see in the usage example, the approach is easy to use and works quite well. I have a couple of qualms with it, however:

• It directly calls TestCase.__init__, which isn't an officially documented feature.
• When different parametrized instances of our test case run, we can't know which parameter was passed. I suppose some hack can be crafted that attaches the parameter value to the test name, but this is very much application-specific.

I'm really interested in feedback on this post. Could this be done better? Any alternative approaches to achieve the same effect?