Today I was happy to find out that Jeff Atwood took my opinion into account when considering the issue and changed the reputation rules on SO to award only +5 for upvotes on questions, including a full re-calc of reputation for all SO users.

Although this decision is going to cost me hundreds of reputation points as I have quite a few highly-voted questions myself, I’m very happy about it. IMHO it will lead to a more balanced reputation system on SO, which is also more correct from the viewpoint of potential employers examining the reputation of SO users. SO has become a phenomenon in the past year or so, and I’m very happy about it. The new reputation rules will make it even better.

Related posts:

1. stackoverflow.com – what the online programming community needs ? I’ve long been claiming that what makes the Perl community...

It’s starting to come back up slowly since Jeff has succeeded recovering data from web caches and with the help of his readers.

But it’s a sobering experience, nonetheless. Placing all the Schadenfreude aside (like most of this Proggit thread), it makes sense to take a moment of pondering to learn about what went wrong and how you can avoid it for your data. Jeff’s hosting – CrystalTech probably had a single point of failure for these websites, and losing a single disk wiped everything out. Jeff had backups, but those were kept on the same server.

My hosting for this blog (and other content, like our photos and travel log) is from bluehost. I remember questioning their support engineer over a chat about their backup strategies when I just started hosting with them. He told me they have backups, but they are all "on-site" – in some kind of a basement somewhere in Arizona. So, although there’s no single disk that can fail, a tornado can probably wipe out my website and many others.

This is why I’m using bluehost’s full backup ability to download a backup of my whole website once in a couple of weeks to my PC. Naturally it also gets to my backup external HD, which I update every week. This distributes the danger over different continents, so I feel much safer. The gzipped backup has reached half a gig by now, but it’s still quick to download over FTP. I’ve even used it myself one time when I deleted something without purpose. Saved me a day or two waiting for bluehost to bring it up from their backups.

Backups are important! Out data is one of the most important assets we have these days, so it’s worthwhile to invest in keeping it safe. You can’t just trust the web-host unless you explicitly pay for a bullet-proof replication service that ensures your data’s safety. Learning from your mistakes is great, but learning from others’ mistakes is much better! So take a few minutes to review the backup strategy for your important data.

Related posts:

1. poor man’s backup At the SW job, all our files are on backed-up...
2. Migrating to Wordpress My blog has just recently migrated from use.perl to Blogger,...
3. A “live” data monitor with Python, PyQt and PySerial The previous couple of posts about the PySerial module for...

Introduction

About a year ago, I taught myself Turbogears (version 1.0.7 at the time) in order to write a small personal web-application for my own use. Although I have very little web development experience, Turbogears was easy to use, and its power was apparent.

Recently I’ve decided to make some improvements to my application. However, in the meantime TG moved on to version 2.0, and because of the many changes it came with, I had two options. Either stick to the old TG version or learn what it takes to use 2.0. Since this is mostly a learning experience, it was a shame to just use an old version. On the other hand, if I need to learn a new approach (for 2.0) anyway, I could learn another framework, while I’m at it.

This is how I got to Django. I made up my mind to rewrite the application from scratch, and to learn the other large Python framework in the process. This post summarizes some of my (highly subjective!) thoughts about it, and how I see the differences between TG and Django.

Related posts:

1. A simple canvas-based Javascript game, with a Django back-end A few years ago I’ve released a clone of the...
2. Deploying TurboGears applications on shared hosting (Bluehost) I’m playing around with web applications now, and my learning...
3. A Python replacement for PHP software ? This blog runs on Wordpress, and I’m quite happy with...

Well, no more! Detexify is a great new service that allows you to “draw” the symbol you’re looking for:

… and it will suggest the Latex code.

Detexify is a learning OCR classifier, and can be “trained” by users to improve it’s performance.

Kudos to the creator of Detexify for a great project. It will definitely be useful…

No related posts.

Wordpress is implemented in PHP, like most popular web applications. In the world of (open-source) web applications, PHP is a 500-ton train sweeping everything else aside. Do you need a CMS ? No problem, have Drupal or Joomla. A photo-album ? There’s Coppermine (which I’m also using for our online photo album). Need a forum ? Everyone knows that phpBB rulez. And some of the big competitors, like punBB and Invision are also written in PHP.

There are literally truckloads of articles online explaining the popularity of PHP, so I won’t just drop another leaf into it. Rather, I just want to express an opinion that it’s a damn shame that this is the situation, because PHP is a bad language that encourages bad programming practice, which makes tinkering with large PHP painful.

I’m a natural tinkerer, and my Wordpress blog is self-customized in several minor ways. I’ve also customized some of the plugins I use for my special needs. I can’t describe the disgust and horror I sometimes experience when fixing or changing something in Wordpress’s PHP code. So much things are global – the variables, the hooks, the functions. Even after PHP added OOP, most code doesn’t use it, or only uses it in a very rudimentary way as module-like wrappers. PHP is probably an ideal language for applications to which many sub-competent hackers add code. It has almost no “accepted style” and people place no limits on what they do. This is a real shame.

I firmly believe that Python is a superior language to PHP. In all domains except web programming it’s also much more popular, so I’m really saddened by the lack of competition to PHP web apps from Python’s side. There’s absolutely nothing in Python that makes it less suitable for web applications than PHP – it’s simply a matter of custom.

Therefore, I think it would be really great if at least one of the big-gun web apps was written in Python. Maybe a competitor to Wordpress, or a forum software. I, for once, would be delighted to use it once it’s stable enough. And it shouldn’t be too hard, either. All it would take is a single dedicated and talented Python hacker, perhaps in a setting like the Google Summer of Code. Take Wordpress for instance. It weighs less than 30 KLOC of PHP code. Even assuming that PHP-Python translation is 1-to-1, brushing some of the more advanced features aside the Python replacement could easily be written in a summer. At first, it can mimic the Wordpress architecture to make the transition easier. Then, once the project gains users and momentum, the code can be slowly and surely refactored to bring more good Python practices in.

I hope someone will start such an endeavor. Perhaps the Django team could offer it as a GSoC project for a Wordpress replacement using Django (which would surely shave lots of code, as WP itself is written in pure PHP w/o any framework beneath).

Related posts:

1. A year with Python Almost exactly a year ago I wrote here about giving...
2. Book review: “Patterns of software” by Richard Gabriel I got to this book from a post on the...
3. Book review: “Rapid GUI Programming with Python and Qt” by Mark Summerfield Mark Summerfield has a lot of Qt experience, having worked...

Options +ExecCGI
AddHandler fcgid-script .py
AddHandler fcgid-script .fcgi

RewriteEngine On
RewriteRule ^(tg_fastcgi.fcgi/.*)$ - [L]
RewriteRule ^(.*)$ /path/to/tg_fastcgi.fcgi/$1 [L]

Problems I ran into

1. Complaints about the version of SQLObject: If you get such complains during the installation or in the logs of the application, install the latest SQLObject with easy_install. This applies to all sub-modules used by TG.
2. Complaints about access_out and debug_out logging handlers. The handlers are defined in your app.cfg file. Make sure that project_module_name in tg_fastcgi.fcgi is pointing to your config directory, where app.cfg will be found.
3. Changing the configuration didn’t affect the application: make sure you kill the FastCGI process to restart the server after each change in the configuration files of the application. pkill tg_fastcgi.fcgi is a useful command. ps uax | grep tg_fastcgi is another.
4. Paths going through tg_fastcgi.fcgi in application URLs: I can access my application with www.mydomain.com/my_app/, but internal links still point to www.mydomain.com/my_app/tg_fastcgi.fcgi/, which works but is less pretty. In many places server.webpath in the configuration file is mentioned as a solution for this problem, but I didn’t manage to get it working without running into permanent 404 errors when accessing the application.
5. MochiKit scripts not working. When you enable the MochiKit widget in app.cfg, TG will insert references to /tg_widgets/turbogears/js/MochiKit.js into your pages. But it doesn’t work for me, somehow. If you’re struggling, you can always copy MochiKit.js from TG’s installation into your project’s directory and point to it directly with a <script src=""> tag.

[1]	As much as I’d like to write about why I chose Turbogears and my experience learning it, I’ll have to leave it for another post.

[2]	For some reason, many guides recommend setting chmod 755 on .htaccess. It works fine with 644 for me, and it seems like a safer option.

Related posts:

1. Installing Python 2.5 on Bluehost Bluehost is my hosting provider for http://thegreenplace.net I’m generally quite...
2. Porting from Turbogears to Django – a personal experience Introduction About a year ago, I taught myself Turbogears...
3. libcollect – collecting Python distributions Here’s an excerpt from the documentation of libcollect: Motivation: Imagine...

I’m generally quite happy with them – the service is stable and the support is responsive. A small annoyance is the old version of Python they have installed by default – 2.3.4. This is quite an old version and many libraries have already dropped support for it.

Luckily, installing a local version of Python is very easy. Here are the few simple steps required to install the latest and greatest Python and run your CGI scripts with it (I’m installing version 2.5.2 in this example).

Access your account with SSH and in the home directory execute:

wget http://www.python.org/ftp/python/2.5.2/Python-2.5.2.tgz
tar xvzf Python-2.5.2.tgz

This downloads and unzips the Python 2.5.2 source distribution. Now install Python locally to your home directory, executing:

cd Python-2.5.2
./configure -prefix=/home/username/python252 --enable-unicode=ucs4
make
make install

Replace username/python252 with your username on the host’s server and the target directory you want to install into. This operation will take a couple of minutes, depending on your server’s speed. It fully configures, compiles and installs Python from sources.

The next step is making the Python you’ve just installed the default Python in your shell. Open ~/.bashrc [1] and add this line at the end:

export PATH=/home/username/python252/bin:$PATH

Save and close the file. New bash shells will now have Python 2.5.2 respond to python. To make it happen in the current shell, type bash, and then python -V to see the new version.

Now, it is important to modify all your Python scripts (including CGI ones) to be executed with your private Python. Modify the shebang line at the top of the scripts to point to it:

#!/home/username/python252/bin/python

That’s about it ! Your Python CGI scripts will now run with Python 2.5.2

Since you’ve made Python 2.5.2 the default in your shell, you can now easily install new Python modules into its site-packages and use them in your scripts. Simply download the modules with wget and install them with python setup.py install. easy_install will work too, once you install it.

P.S. I expect this method, perhaps with minor modifications, to work for other providers as well, and not only Bluehost.

Related posts:

1. Deploying TurboGears applications on shared hosting (Bluehost) I’m playing around with web applications now, and my learning...
2. Creating Python extension modules in C I’ve successfully created a C extension for Python, basically following...
3. Setting up an initialization file for CLISP on Windows CLISP, like other CL implementations, loads an initialization file on...

The Perl script I’ve previously written to answer this question has stopped working, because Google changed the layout of their result HTML [1].

Instead of fixing the Perl script, I’ve decided to rewrite it in Python, and at the same time check what other options I have to complete this task.
It turns out that in the time that has passed since my previous research on this topic, Google have added an option to use their AJAX Search API with external HTTP requests. Just build a GET request and the results will be resulted as a JSON string. Undoubtedly, this is more convenient than screen scraping.

However, this approach is still limited – because the API won’t ever return more than 32 results (and even for that, 4 separate requests have to be issued) [2]. To get more results, one still has to use screen scraping.

Anyway, I’ve coded both approaches as Python modules. The first one, google_api_search uses the Google search API to find results. The second, google_web_search issues normal HTTP search requests to Google with mechanize, and processes the results with BeautifulSoup. I’ve also whipped a simple CGI interface to these modules: google_search_rank.

Related posts:

1. How high on a google search does your website rank ? Update 13.09.2008: The script described here no longer works. See:...
2. Search keywords in FireFox Firefox allows to conveniently add “keywords” to search boxes, to...
3. google AdSense anecdote on perl6 compiler list Google AdSense is a good idea. A really really good...

Executive summary: I don’t like mailing lists. I think they’re a thing of the past, and these days using a web interface like a forum or a Google Group makes much more sense. So, it makes me angry that a lot of groups I’m interested in lately hold their discussions in mailing lists.

Mailing lists used to be popular some years ago. Sure, when web access was via a modem and you only connected for an hour a day, it was convenient to download all the mail to your PC and read it offline.

But today, with fast, permanent web connections for almost everyone, I think mailing lists are an anachronism. Forums and places like Google Groups are much better – they present all the information with full archives in a single place, you don’t have to fill your mailbox with all the list’s messages, and it’s easy to post and view discussions in hierarchical threads.

With a mailing list that delivers dozens of mails per day, you have to have an inbox full of mailing list correspondence, have no good search possibilities and can’t see thread hierarchies – everything is flat. So why is it good, why many groups still prefer this discussion method ?

Specifically, I fume about Python. Although its main mailing list is reflected to comp.lang.python, most of the libraries and topical discussion groups use mailing lists, to which you have to subscribe. So let me get this straight. I’m interested in distutils, py2exe, matplotlib, wxPython and several other libraries. Does it really mean I have to subscribe to a gazillion mailing lists, just to go and ask a question / search for answers in one of them, once in a blue moon ? What other option to I have if I want to be able to post ?

Sure, there are sites like Nabble and Gmane, that reflect mailing lists to hierarchical forums bi-directionally, and they are great. I personally use Nabble for most of my participation in Python-related lists.

But some mailing lists haven’t registered with such services, so all that’s left is using the mailing list itself. In particular, I’m fuming about python-es, a mailing list for Python in the Spanish language, which isn’t connected to Nabble and is already registered in Gmane as read-only. Argh, this is annoying.

Related posts:

1. spam All the messages in my mailbox today were Spam. This...
2. Rant about Common Lisp and implementations In time I became very frustrated with my educational path...

For example, go to Google, right-click on the search box and select “Add a keyword for this search”. Give it a name and a keyword, for example “g”.

Now, you’ll be able to just type “g whatever” at the address bar and Firefox will execute a Google search on “whatever” and show you the result. Combined with the Alt-D hotkey that places the keyboard cursor in the address bar, this is a nice productivity boon.

So far I’ve added keywords for Google, Wikipedia, Babylon (translations), and my own website (sometimes I look for stuff I’ve written on before).

Thanks to Misha for the tip.

Related posts:

1. firefox After a few weeks of playing around, I decided to...
2. How high on a google search does your website rank ? Update 13.09.2008: The script described here no longer works. See:...
3. Great Firefox productivity tip One of my heaviest uses of the mouse is browsing...