It’s time for Python 2.7

October 9th, 2011 at 11:24 pm

Two years ago I decided to move my main Python development from version 2.5 to 2.6, and wrote about the new features of 2.6 which I found most useful.

Now I think it’s a good time for me to move to 2.7, so this post has a similar goal.

Python 2.7 is not an ordinary version. It’s the last major release of the 2.x series – no more language features and new libraries will be added to Python 2.x – all new development is focused on 3.x. However, since the Python core development team recognizes that Python 2.x is going to be used for a long time yet (even if just for legacy systems), the 2.7 release will be in maintenance mode for longer than usual, and there’s a real effort to fix any bugs found in it.

Python 2.6 inherited some features from 3.x that allowed easier transition in the future (for example, it’s not very hard to write code that runs on both Python 2.6 and 3.x). Version 2.7 takes this another step forward, adding even more features from 3.x, which both make the future transition even easier, and improve the language overall. Here are some examples:

set literals and comprehensions

Thanks to the new set literal syntax, the following two statements are equivalent:

>>> myset = set([2, 3])
>>> myset = {2, 3}

Now sets are as "built-in" in Python as lists and dictionaries are, and this is a good thing, because sets are a very convenient data structure for many different purposes. It’s not that sets weren’t very usable in previous versions – but their own syntax gives them a "first-class citizen" status.

Additionally, set and dict comprehensions are now possible:

>>> {i*2 for i in range(5)}
set([0, 8, 2, 4, 6])
>>> {i: i*2 for i in range(5)}
{0: 0, 1: 2, 2: 4, 3: 6, 4: 8}

List comprehensions make a lot of problems easy to express succinctly. Dict and set comprehensions further improve Python in this respect.

unittest enhancements

A very welcome new feature is some important additions to the unittest module. Unlike most parts of the standard library, unittest is something I use in virtually every project, so any improvement in it matters a lot. Some highlights:

  • unittest now has simple test discovery capabilities. This means you can run it (by invoking the module with python -m unittest) on a directory and it will automatically discover and execute all test files in it. This is a feature other unit-testing libraries (like nose) boast, and it’s great to have it built-in.
  • assertRaises can now work as a context manager, which makes it much more pleasant to use.
  • Resource allocation and cleanup for tests should now be much easier to implement, thanks to new class and module-level setUp and tearDown methods, and a method for adding additional custom cleanup functions.
  • A lot of useful new assertion methods have been added. For example (all of these also have negative variants): assertIs, assertIn, assertIsInstance, assertRaisesRegexp.
  • assertEqual became much smarter about reporting failures when comparing lists, dicts, tuples and sets. For example, if two lists aren’t equal, only the mismatch is reported instead of dumping them wholly as the old assertEqual would do.

No more cStringIO

StringIO is a great tool, but it’s written in pure Python and thus is slow, which can limit its usefulness when performance is critical. So for a long time we’ve been used to importing it from the alternative C-based package cStringIO.

In Python 2.6, the io package from 3.x has been backported, but with an old and inefficient pure-Python implementation of StringIO. Finally, Python 2.7 brings the C-optimized io package from 3.1, and cStringIO is no longer needed (it still exists, of course). Just:

>>> from io import StringIO

And you’re good to go. This will work in Python 3.x too, of course, so one less compatibility issue to worry about.

Note that in Python 3.x, I/O handling has been considerably redesigned, in part to accommodate for the stricter distinction between textual and binary data that 3.x makes. The io module backported to 2.x allows to use this code, although it’s not the default for file and stream handling (i.e. the built-in open function won’t use it). When applicable (such as with StringIO), it’s recommended to use the io module in 2.7, for easier future transition to 3.x.

OrderedDict

Python prides itself at being a "batteries included" language, but naturally you can’t have all possible batteries included from the start, so the standard library slowly grows over time, adding more and more useful tools.

One tool that has been on the "wanted" list for a long time is an ordered dictionary – a dictionary that remembers the order in which the items were inserted into it. So this data structure appeared in many different recipes and forms, "unoficially". Python 2.7 adds the collections.OrderedDict class for this purpose, so that’s another battery now included in the standard library.

argparse

Not satisfied with the two existing tools for parsing command-line arguments (getopt and optparse), Python 3.2 added another one – argparse, an enhanced version of optparse (which is also why optparse became deprecated). Python 2.7 takes this module from 3.2. So if you feel optparse is too limiting for your needs, you will be happy with this change.

There are many more changes in 2.7, the above is just a sample of what I found to be interesting. For the full list see the What’s New in Python 2.7 documentation page. Overall, it’s a pretty good update, making Python coding even more pleasant.

I want to stress again the importance of the 2.7 release, being the last in the 2.x series. That’s it, the last incremental upgrade. Consider that while 2.7 borrows features from 3.2 and 3.1, when 3.3 is released there will be no corresponding 2.x release to backport its features to. IMHO this is when Python 3.x will finally start growing really apart from 2.x (in terms of features), and this will hopefully encourage more people to do the switch. For now, my recommendation is to make sure your code runs on both 3.x and 2.x. With 2.7, it isn’t very hard to achieve.

Related posts:

  1. Python development – improving ElementTree for 3.3
  2. Creating Python extension modules in C
  3. pyelftools – Python library for parsing ELF and DWARF
  4. Python documentation annoyance
  5. New year’s Python meme 2011

7 Responses to “It’s time for Python 2.7”

  1. BernardoNo Gravatar Says:

    There’s a problem with “make sure your code runs on both 3.x and 2.x”.
    I often write my code with Python 3.2 and make it work in 2.7 & 2.6… The problem lies in the bytes/chars change and I never know the proper solution. Because of the languages I support, need a lot of unicode to work properly…

    I’ve tried:
    1 – __future__ unicode_literals (breaks a lot of modules)
    2 – Define a new str to accept encodings as second argument (pretty ugly approach)
    2.1 – use the .encode() method of str (available on 2.x for forward compatibility and seems a better approach)
    3 – Check the interpreter version inside some functions that can mess with that (extra coding)

    I’ve seen some people saying that you should make a Python2 version that can be translated to Python3 only using 2to3.py… I don’t like the approach (I want to get rid of Python2 ASAP) but seems to have a lot of followers.

    What do you do?

  2. elibenNo Gravatar Says:

    Bernardo,

    Yep, the binary/unicode duality is probably the most difficult aspect of keeping the code running on both 2.x and 3.x, because it’s the biggest and most fundamental change in 3.x

    Myself I haven’t had to deal with such code a lot, but there are some good references on this issue. For example, in the six package (http://packages.python.org/six/) and the “Porting to Python 3″ book (http://python3porting.com/)

  3. Thoomas WaldmannNo Gravatar Says:

    I recently considered raising requirements for moin2 to python 2.7, but I didn’t do it.

    Debian Squeeze does not have 2.7, not even in backports.

    Debian Wheezy (next stable) has it, but still maybe 2 years away.

    Same thing might be true for some other linux distributions.

    So I kept py 2.6 for moin 2.0 and will reconsider raising in 2y.

  4. elibenNo Gravatar Says:

    Thoomas,

    I find it strange that they’re taking so long. After all, 2.7 was released more than a year ago and has already went through 2 updates; it’s not as if it’s likely to break existing 2.6-targeted code.

  5. Jacobo de VeraNo Gravatar Says:

    I got excited about the test discovery feature and went ahead to try it. However, it seems you need to use the discover subcommand, as described in the docs.

    Thanks for sharing this, I had completely missed this feature.

  6. Everett GammelNo Gravatar Says:

    I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end. I would like to read newer posts and to share my thoughts with you.

  7. Thomas WaldmannNo Gravatar Says:

    Just found my own old post again while doing some refactorings after raising requirement to 2.7 – now that debian wheezy is released with python 2.7.

    Luckily it took only 1.5y, not the estimated 2y. \o/

Leave a Reply

To post code with preserved formatting, enclose it in `backticks` (even multiple lines)