A year with Python

General impression

My thoughts on Python are not much different from the short summary I posted just a few weeks after starting with the language. If anything, the small deficiencies "grew on me" - I got used to the couple little quirks in the syntax, and found out that the other things are not so bad.

For example, although Python doesn't have CPAN, I haven't yet encountered a need for which no Python library has been written. In this aspect the last year, at least in my view, has shown a trend towards Python from other languages. Google and other companies are releasing Python-only APIs, for instance. And I think it's hard to find another language with two such great bindings to complete GUI frameworks like PyQt and wxPython.

I also lamented the clutter in Python's standard library. Well, that is being fixed with Python 2.6 and especially 3.0, which will become mainstream in a few months.

Regarding Python's pluses, the significant whitespace turned out to be a much bigger one than I've ever expected. Over the last year I've read the code of many libraries, and almost always found it quite readable and simple to follow. Python's PEP-8 is a good set of rules that's followed to some extent by virtually everyone in the community, which adds a lot. Company-wide coding standards exist for a reason, and PEP-8 is a community-wide coding standard.

Moreover, I've taught Python to a few junior engineers and interns at work - people with little programming experience. I was amazed at how easy their code was to understand. I've had the "pleasure" to read Perl code written by programming newbies, and it's a painful experience I would never want to go through again. Python is different.

Python-mania

Python is taking over practically all my programming. Over the past year, it has completely replaced Perl and Ruby. It has also somewhat shoved C and C++ out of the picture, because Python has such great bindings to existing C code (ctypes), and also because it has great libraries for practically everything.

But the most surprising effect of Python on me is that it's slowly but surely pushing Lisp out of my mind. Back when I was programming in Perl, I kept coming back to Lisp from time to time, trying to make the shift, not succeeding and going back. I guess I just felt unsatisfied with what I had.

With Python, this is just gone. I'm fully satisfied with this language. I've very glad that I learned the functional programming paradigms from Lisp, and I use them daily in Python. But overall, Python is simply a better language. Moreover, it has a much larger ecosystem - libraries, development community, bindings, and support from large companies.

Getting things done

I did a rough count of the amount of Python code I've written this year. I count commented, tested lines of code, meaning that comments and unit tests are included in the count. The result is roughly 40 KLOC, which isn't bad at all.

Here's a partial list of stuff I've implemented with Python so far. Some of it at work, some at home:

• A couple of simple web-applications for my own use. One using plain CGI, and the other using the TurboGears web framework.
• pycparser - a complete parser for ANSI C90.
• Parts of a VHDL parser. This is still work in progress.
• A converter between binary data description formats, such as Intel HEX.
• Several versions of a recursive-descent parser for a simple language
• Many scripts for Scite
• At work I've developed a framework for using Python for verification of VHDL models. Python is used to write the test scenarios and generate VHDL testbenches from them.
• A few applications for controlling embedded hardware via the serial port (with elaborate wxPython & PyQt GUIs)
• A few applications for analyzing test reports and building summaries in a user-friendly format
• A complex multi-threaded web-page download and analysis program for my own use
• A couple of games: a Tetris clone with wxPython and a small game with Pygame as part of a tutorial
• Analysis of numeric data with matplotlib
• Parts of an assembler and linker for a synthetic language
• Solved dozens of Project Euler problems, all in Python.

Most of these projects were covered at least to some extent in this blog, and can be found by searching. If something interests you and you can't find it in the blog search box, drop me an email.

Libraries

Finally, I want mention a few Python libraries I've found especially good and/or helpful in my work with Python this past year. In no particular order:

• docutils & reST (reStructuredText): a great collection of libraries and utilities for formatting text
• matplotlib: Matlab-like plotting
• PLY: a Python implementation of Lex & Yacc. It's very intuitive to use, powerful and useful for complex parsing tasks.
• py2exe: a wonderful .exe packer that works with all major Python libraries and produces fast-loading executables
• PyQt: a great binding to the powerful Qt framework - my current choice for GUI programming in Python
• wxPython: another good GUI-binding
• PyYAML: a simple-to-use YAML lib. I wish it were part of the standard Python library.
• pyserial: the best scripting-language binding to the serial (COM) port I've seen anywhere
• BeautifulSoup: a bit cranky but overall an excellent way to retrieve data from HTML pages
• ctypes: calling C DLLs has never been so easy!

To the creators of these libraries - well done! May others learn from you on creating useful, high-quality and well documented code.