Compiling Python extensions with distutils and MinGW

Tags Python

Yesterday I wrote about compiling Python 2.5 extensions using MS Visual Studio. However, I also wanted to make it work using distutils (as described here), which appeared to be the simplest method.

Trying distutils with Visual Studio 2005 Express

I have the free Visual Studio 2005 Express installed. I used it to successfully compile Python from sources, and wanted to tie it to distutils to compile the extension. Unfortunately, distutils is very strict about the MSVC version you compile extensions with, and complained:

running build
running build_ext
error: Python was built with Visual Studio 2003;
extensions must be built with a compiler than can generate compatible binaries.
Visual Studio 2003 was not found on this system. If you have Cygwin installed,
you can try compiling with MingW32, by passing "-c mingw32" to setup.py.

I consulted the Python IRC channel, and got a warm recommendation to use MinGW anyway, because many people has tested its compatibility with the Python distribution.

distutils with MinGW, first attempt

I've downloaded the latest version of MinGW, and tried to call distutils with the appropriate flag:

python setup build -c mingw32

But got this error1:

running build
running build_ext
building 'hello' extension
creating build
creating build\temp.win32-2.5
creating build\temp.win32-2.5\Release
d:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python25\include -IC:\Python25\PC -c hellomodule.c -o build\temp.win32-2.5\Release\hellomodule.o
writing build\temp.win32-2.5\Release\hello.def
creating build\lib.win32-2.5
d:\mingw\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.5\Release\hellomodule.o build\temp.win32-2.5\Release\hello.def -LC:\Python25\libs -LC:\Python25\PCbuild -lpython25 -lmsvcr71 -o build\lib.win32-2.5\hello.pyd
build\temp.win32-2.5\Release\hellomodule.o:hellomodule.c:(.text+0x3e): undefined reference to `_imp___Py_NoneStruct__abENT__apos;

build__abENT__#92;temp__abENT__#46;win32-2__abENT__#46;5__abENT__#92;Release__abENT__#92;hellomodule__abENT__#46;o:hellomodule__abENT__#46;c:(__abENT__#46;text+0x46): undefined reference to`_imp___Py_NoneStruct'
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

This got me stumped, because I could find nothing useful about this error in Google. However, I did get to this page, where the solution was found in the comments.

The solution

Apparently, the Python export lib (libs/python25.lib in the Python distribution) is not good enough for MinGW, and it wants a Unix-y .a file to list the exports of Python's DLL (the one where the interpreter itself lives).

So you have to create libpython25.a file for the Python library. If you want, download the one I've created and skip all these steps. It should work with Python 2.5. (Update 25.11.2009: here is the lib for Python 2.6)

To create libpython25.a on your own (taken from here with minor modifications):

  1. Find python25.dll (it will probably be in windows/system32)
  2. Now you have to list the exports from this DLL. Several tools can do this (including dumpbin), but pexports is the most convenient for this task. Get it from here (I'll keep this link alive for a long time, promise).
  3. Run pexports.exe python25.dll > python25.def
  4. Now, if you've installed MinGW and added its bin directory to your path, you should be able to just call dlltool (a utility that comes with MinGW): dlltool --dllname python25.dll --def python 25.def --output-lib libpython25.a
  5. This has created libpython25.a

Now just place libpython25.a in the libs directory of your Python distribution, and run distutils again2:

running build
running build_ext
building 'hello' extension
creating build
creating build\temp.win32-2.5
creating build\temp.win32-2.5\Release
d:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python25\include -IC:\Python25\PC -c hellomodule.c -o build\temp.win32-2.5\Release\hellomodule.o
writing build\temp.win32-2.5\Release\hello.def
creating build\lib.win32-2.5
d:\mingw\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.5\Release\hellomodule.o build\temp.win32-2.5\Release\hello.def -LC:\Python25\libs -LC:\Python25\PCbuild -lpython25 -lmsvcr71 -o build\lib.win32-2.5\hello.pyd

Big success ! You can now use the extension from your Python code.

Updates

  • It has been mentioned in a couple of places that with the latest MinGW the .a file is no longer needed and it's enough to place the Python DLL in libs. I'm using the latest MinGW, ActiveState Python and distutils, and it didn't work for me that way.

1 I'm using a trivial sample extension called 'hello'. Here's its C file:

#include 
 
static PyObject* say_hello(PyObject* self, PyObject* args)
{
    const char* name;
 
    if (!PyArg_ParseTuple(args, "s",&name;))
        return NULL;
 
    printf("Hello %s!\n", name);
 
    Py_RETURN_NONE;
}
 
static PyMethodDef HelloMethods[] =
{
     {"say_hello", say_hello, METH_VARARGS, "Greet somebody."},
     {NULL, NULL, 0, NULL}
};
 
PyMODINIT_FUNC
 
inithello(void)
{
     (void) Py_InitModule("hello", HelloMethods);
}

And here is the distutils setup file:

from distutils.core import setup, Extension
 
module1 = Extension('hello', sources = ['hellomodule.c'])
 
setup (name = 'PackageName',
        version = '1.0',
        description = 'This is a demo package',
        ext_modules = [module1])

2 Note that you can run either setup.py build -c mingw32 or setup.py build --compiler=mingw32 - they are suggested in different places, but actually mean the same.

For comments, please send me an email.