Helping improve the documentation of Python

July 12th, 2011 at 9:28 am

As I was reading this discussion on the drawbacks of Python, I was greatly surprised to see that a few answers, including the currently top-rated one, mentioned Python’s documentation as one of its weak sides.

I have always considered Python’s documentation to be excellent. I also know that a lot of effort is being invested in keeping Python’s documentation correct, useful and up-to-date. A sizable part of my own work as a Python core developer so far was focused on documentation.

To make this post constructive, I propose the following:

If you find some flaw in Python’s documentation, don’t just sit there fuming. The Python developers really care about documentation, and will gladly work towards making it better. You have a few options to influence:

  1. Open a documentation issue on the Python issue tracker.
  2. Send an email to the docs@python.org mailing list.
  3. Or just shout it out loud (in a blog, a forum, Stack Overflow, or wherever), hoping someone will pick it up.

You can also send me an email – I’ll be happy to discuss and re-route the report to the appropriate destination.

P.S. While in my Python work I focus on CPython – the official C implementation, documentation is one of those areas that can benefit all implementations of Python, since they all basically implement the same "language spec" (barring some slight differences such as the ones documented here). The CPython documentation serves as the formal description of the language and its standard library. So helping make the docs better will not only help CPython, but Jython, IronPython and PyPy as well.

Related posts:

  1. Python documentation annoyance
  2. Python development – improving ElementTree for 3.3
  3. Python development switches to Mercurial source control
  4. How to misunderstand security
  5. Bob: a Scheme interpreter, compiler, and VM in Python

20 Responses to “Helping improve the documentation of Python”

  1. BernardoNo Gravatar Says:

    > since they all basically implement the same “language spec”

    The PyPy’s compatibility webpage says that there’s no documentation for when a overhiden method of a subclass of a built-in get called.

    http://pypy.readthedocs.org/en/latest/cpython_differences.html#subclasses-of-built-in-types

    Before reading that I didn’t knew why some code I wrote wasn’t working…

  2. elibenNo Gravatar Says:

    Bernardo,

    Thanks, I’ve added a clarification to that sentence.

  3. jimdNo Gravatar Says:

    I also think the search engine produces poor results.
    Of course someone could just use Google but some kind of simple rating giving priority to title, label matches would be nice.

  4. JoeNo Gravatar Says:

    The top rated answer has 20 votes, but it has the following comment with 19, which essentially negates the bullet point about documentation:

    “Python docs are some of the best. Hands down better than Java.”

    I too find it surprising that the OP says Java and PHP have better documentation. Can they type “help(almost_anything)” and get introspected documentation in Java or PHP?

  5. Mikko OhtamaaNo Gravatar Says:

    Python documentation is bad, compared to any other language besides C man pages. It’s not bad per se compared to e.g. average open source project documentation, but for a proramming language it’s not that great.

    Most of the documentation (not all of it) is written by professionals for professionals – not for novices. Docs make the assumption that you know UNIX, underlying C architecture, system calls and so on. The docs assume that you have graduaded in CS and you have done your 101 of OS architectures and so on.

    There are no cross references or see also: “you might actually want this function”.

    Functions lack examples: if you look the page http://docs.python.org/library/os.path.html you see zero full code example. You can compare it to this http://php.net/manual/en/function.pathinfo.php as an example. PHP docs have much more novice friendly attitude.

    I also believe that the documentation won’t be improved by call for arms. If you want to improve the documentation you must have a tool in the documentation itself to give feedback and improve it. This is not 80s and LaTeX anymore. People are used to Facebook Like button. Improving documentation should be as easy as pressing Like button.

    I think the greatest near term thing what could happen for Python is that it would get PHP-like per function discussion board where people can post examples and ask for help.

    A similar idea was proposed for Plone community, but the discussion ended like “let’s keep docs separate from the discussion as no one is able to answer the support calls or the docs would include off-topc discussion” (actually plone.org documentation section has its commenting disabled)

    I think that’s shitty attidude.

    Python docs would be the best place to let people share their worries and the best feedback channel to actually improve the documentation.

    Some more ideas here:

    http://blog.mfabrik.com/2011/03/29/when-python-sucks-how-you-call-a-function-and-document-it/

  6. Mikko OhtamaaNo Gravatar Says:

    … and even if you tried to improve documentation you need a different mindset to write documentation different people find useful. It takes practicing to see things from newcomer viewpoint (empathy and all, the skills engineers often lack).

    I don’t claim the documentation is a hard task, but even if some hardcore Pythonist wanted to improve the documentation the result might not be beneficial for the great masses.

  7. Marius GedminasNo Gravatar Says:

    The documentation available online is better than the documentation accessible via pydoc. My beef is that it’s harder to locate the page you want, compared to pydoc module.function.

    I actually came here to complain about the links to http://docs.python.org/ that pydoc module displays in the “MODULE DOCS” section being broken, but today they all work for me. I wonder if that was a transient error on the python.org side. Or maybe I was just looking at the more obscure modules — try following the module doc link that pydoc xml.etree.ElementTree gives you, for example.

  8. Nick CoghlanNo Gravatar Says:

    The missing piece in the official documentation is one that sits in between the library reference and the language reference to explain to people how it *works* (and not in a language implementor focused way).

    A version of this exists and has been contributed to the PSF (by me), but I need help getting it out of the ODF files and updated for current versions of Python: http://svn.python.org/view/sandbox/trunk/userref/

  9. Nick CoghlanNo Gravatar Says:

    On reading that page, I’m now wondering if the people griping about the poor docs haven’t looked at them since they were switched over to Sphinx in 2.6.

    The gripes are significantly more valid when it comes to 2.5 and earlier (compare http://docs.python.org/release/2.5.4/ to the current docs)

  10. Nick CoghlanNo Gravatar Says:

    pydoc definitely has had its problems – it received a major update in 3.2, but that won’t help anyone still on 2.7.

  11. elibenNo Gravatar Says:

    Marius,

    Which version of pydoc are you talking about here?

    Also, if you find that links are incorrect this is surely a bug (which may be, as you noted, an intermittent connection/server error), so feel free to report it to the bugs tracker / mailing list.

  12. elibenNo Gravatar Says:

    Nick,

    I doubt there will be a lot of motivation to invest considerable effort into improving the documentation of 2.7 – perhaps better documentation should be one of the selling points of 3.x :-)

  13. Nick CoghlanNo Gravatar Says:

    Mikko’s feedback regarding examples is exactly why some of us wanted to automatically include direct links to Python Module of the Week (http://www.doughellmann.com/PyMOTW/, aka “Python Standard Library By Example”) from the official docs. Let the official docs continue to serve as the library *reference*, but provide quick access to additional explanatory material.

    I also recently discovered http://homepage.mac.com/s_lott/books/python/html/index.html as another potentially useful resource for potential Python programmers.

    Rather than trying to do everything ourselves, we may want to try to do a better job of directing people towards resources that *we* think do a good job of introducing people to Python. We trust committers to actually make changes to the code and documentation, we should trust ourselves to recommend good web resources, too.

  14. elibenNo Gravatar Says:

    jimd,

    Can you be more specific? It would help if you could mention a few terms where you got bad search results.

    Miko Othamaa,

    You are raising some interesting points, but why the pessimism :-) ? The Python community definitely wants the language to be welcoming to newbies (and in fact Python is widely acclaimed as on of the best languages to start programming with). However, there are of course tradeoffs to consider. For instance, how “dumbed down” and lengthy the docs should be is a tradeoff. After all, there are many great free books about Python out there that take care of this. The reference documentation should also cater to the more experienced devs, treating them with due respect, and not explaining trivial facts in minute detail.

    That said, I repeat that I find your notes definitely worthy of consideration and I will consider raising some of them for discussion.

  15. elibenNo Gravatar Says:

    Nick,

    While PyMOTW is great, there’s a version problem. For instance, it now contains Python 2.x code. Can you link that from the 3.x docs? I don’t think so, because those newbies needing the examples will just get syntax errors which will make them even less pleased.

    A alternative idea would be to selectively fold parts of PyMOTW into the documentation. I’m sure Doug will have no problems with it, and it shouldn’t take much effort since the information is already there.

  16. Mikko OhtamaaNo Gravatar Says:

    Eli:

    I am pessimistic because I have been working with great Python developers for too long ;) I know their mindset and how hard it might be to change. But the starting point is admit the current state of affairs and if we need to do something.

    Cheers

  17. Henri de FeraudyNo Gravatar Says:

    I think Mikko Ohtamaa says it very well. I’ve been working with Python for a very long time and it has hardly been improving.

    I worked in technical writing for years and I would not have been proud of the Python documentation.
    There are very few examples, There are very few hyperlinks.
    There should be a distinction between what is to be typed literally and what is just an example. And the notation should be summarised on every page, or a link should be provided.

    Take the ftplib documentation for example:

    http://FTP.storbinary(command, file[, blocksize, callback, rest])
    Store a file in binary transfer mode. command should be an appropriate STOR command: “STOR filename”

    well “filename” should have been in italics for consistency.
    Why isnt there are list of FTP commands somewhere, even with a warning that it’s incomplete?

    consider the doc for transfercmd: we are told we could use EPRT or PORT without any explanation of how they differ. What about the difference between EPSV and PASV? Is there a difference? The doc doesnt say.

    Other gripes:
    templates : this really could do with examples! People dont want to read vague explanations when
    examples say it better.

    string encoding : this could do with a very detailed beginner friendly explanation.

  18. elibenNo Gravatar Says:

    Henri,

    It’s good that you’re providing concrete examples. If you open issues on them in the Python bugs database and set it to documentation, I’m sure the problems will be fixed. Lately there’s been more interest in improving the Python documentation.

  19. Ufuk KirikNo Gravatar Says:

    I totally and full-heartedly agree with Mikko Ohtamaa on the drawbacks of Python standard library. I think he has very valid points and particularly good point-of-view. Allow me to elaborate:

    I am not a CS major, I did applied mathematics instead. I also do not work as a professional programmer, I do biomedical research. Here’s the kicker though, I have put quite a few hours into programming both during university, and even now at my current work, every single day. I have poked around in Visual Basic and Pascal during high-school, Perl/PHP/SQL during college, settled for Java even though I coded in C and MATLAB as well. What I am trying to say here is that, programming is not exclusive to hardcore programmers anymore, thus you need to keep that in mind when designing documentation.

    Now recently I started playing around with Python, in search of a programming language in which I can minimize the development time for small tools (like file parsers and quick scripts). I have to say coming from Java ecosystem, Python documentation is simply undecipherable. I do not mean any offense to anyone who have put countless hours into the project but the doc pages are neither aesthetically nor pedagogically well designed. Good reference pages should aim to minimize the time and effort it takes for someone to find what he/she’s looking for. More often than not, I find myself wondering “OK I have got X, what can I do with it?” (or alternatively phrased “I have X, how can I get what I want with it?”)

    Unless you know your standard library well, finding the answer to that question is surprisingly hard with Python. Now admittedly the help() function is fantastic and the fact that one can try out one-liners in the interpreters is just sweet! But again given at least a couple of hundred lines of code, the fact that I can run quick and dirty tests are usually not what I need. Instead I would like to look at what properties and methods I have to play with given an object/class. I realize that this is a very Java:ish way of looking at things, and that Java language had corporate backing. Nevertheless many students start coding with Java during their university years, and with Android market growing as it is, one cannot deny the amount of “new” Python users, that will have a similar background to mine.

    Long story short, the fact that information on the doc pages is correct doesn’t mean anything, as I would expect the information to be correct in a reference. What makes a good reference in my opinion is the ease to find and digest new information, in the shortest time possible. On this regard Python is way behind other major programming languages.

    I really hope that you, and other Python developers take this as a constructive critique and not simply a whining newbie, and that Python documentation is improved in the near future, because writing code in Python is fun and I would like to be able to continue using this cool language :)

  20. elibenNo Gravatar Says:

    Ufuk,

    Thanks for the thoughtful comment.

    I don’t want my post to be misunderstood. I think a lot of the commenters just refer to my claim that I personally find the Python documentation pretty good. But that was not the point of my post :-)

    The point, rather, is that help is wanted and if you have specific qualms with the documentation, please report them in one of the indicated methods, and there’s a good chance these qualms will be fixed. Reporting such problems is a great way to contribute to the overall quality of Python.

Leave a Reply

To post code with preserved formatting, enclose it in `backticks` (even multiple lines)