ctypes - calling C/C++ code from Python

Introduction

A couple of years ago, I wrote about compiling C DLLs and using them from Perl code. Today, Python is my language of choice, and I want to write about how to achieve the same in Python.

It turns out that (like many other things) using C/C++ DLLs in Python is much simpler. ctypes - the Python module that implements it, is one of the best designed and documented foreign function interfaces (FFIs) I've seen in any language. And true to the 'batteries included' dogma, it is part of the Python standard library (from version 2.5).

Simple types and buffers

Here's a sample function written in C and compiled into a DLL. The header file [1] :

#define DLL_EXPORT __declspec(dllexport)

DLL_EXPORT int __stdcall test_buf(char* buf,
                                  int num,
                                  char* outbuf);

Here's the implementation:

int __stdcall test_buf(char* buf,
                       int num,
                       char* outbuf)
{
    int i = 0;

    for (i = 0; i < num; ++i)
    {
        outbuf[i] = buf[i] * 3;
    }

    return num;
}

Now, here's how to call this from Python using ctypes:

from ctypes import cdll, windll, c_long, c_int, c_char_p, create_string_buffer

# Use cdll for functions compiled with __cdecl
#
libc = cdll.msvcrt
print "The time() is: " + str(libc.time())

# Use windll for Windows API and functions
# compiled with __stdcall
#
test_dll = windll.dll_test

# Load the function test_buf from the DLL
test_buf = test_dll.test_buf

# Create a pointer to a Python data buffer
data_in = c_char_p('\x04\x21\x41\x1F')

# Allocate space for the output buffer
data_out = create_string_buffer(4)

# A 'long' object
numbytes = c_long(4)

# Finally, call the function test_buf, passing it the prepared
# parameters and receiving the return value
#
ret = test_buf(data_in, numbytes, data_out)

# Inspect the results
#
import binascii
print "Returned", ret
print "Out =", binascii.hexlify(data_out.raw).upper()

Callbacks

ctypes can also gracefully handle callback functions (a non trivial task for FFIs). Here's another C function compiled into the DLL:

DLL_EXPORT int __stdcall test_cb(void (*fp)(int),
                                 int arg);

With a trivial implementation that's enough to demonstrate what we need:

int __stdcall test_cb(void (*fp)(int),
                      int arg)
{
    fp(arg);
}

And here's the Python code to call it:

from ctypes import windll, c_int, CFUNCTYPE

test_dll = windll.dll_test
test_cb = test_dll.test_cb

# Define a callback function type, as a function that returns
# void and takes a single integer as an argument
#
CB_FUNC_TYPE = CFUNCTYPE(None, c_int)

def foo(arg):
    print 'foo Called with', arg

# Wrap foo in CB_FUNC_TYPE to pass it to ctypes
cb_func = CB_FUNC_TYPE(foo)

# Finally, call test_cb with our callback. Note the printed
# output
#
test_cb(cb_func, 10)

Note that I've used the CFUNCTYPE function to create the callback prototype. This tells ctypes that the callback will be called using the standard C calling convention. This is because I've specified no convention when declaring void (*fp)(int). Had I declared test_cb as:

DLL_EXPORT int __stdcall test_cb(void (__stdcall *fp)(int),
                                 int arg);

I would have to use WINFUNCTYPE instead of CFUNCTYPE (the rest being exactly the same).

The lesson from this is simple: while you're quite free to define any calling convention as long as it's your code at both sides of the call, be careful to notice the calling conventions for all functions and callbacks in 3rd party C/C++ code you want to call from Python.

Conclusion

Python's ctypes module is a very powerful FFI tool. It supports all the complex features [2] you might need when wrapping DLLs written in C/C++. You are encouraged to use it for both 3rd party C libraries and for extending your Python code with some C for performance. Using ctypes for this task is much more simple and flexible than writing full-fledged Python extensions.