Flexible runtime interface to shared libraries with libffi

Calling code from shared libraries in C is simple with dlopen / dlsym (LoadLibrary on Windows). I provided a comprehensive example in the article on Plugins in C; here, I'll start with a simplified example.

Here's a sample C library compiled into libsomelib.so. First, the header file somelib.h:

#ifndef SOMELIB_H
#define SOMELIB_H

typedef struct {
    int num;
    double dnum;
} DataPoint;

DataPoint add_data(const DataPoint* dps, unsigned n);

#endif /* SOMELIB_H */

And the implementation, somelib.c:

#include "somelib.h"

DataPoint add_data(const DataPoint* dps, unsigned n) {
    DataPoint out = {.num = 0, .dnum = 0.0};

    for (unsigned i = 0; i < n; ++i) {
        out.num += dps[i].num;
        out.dnum += dps[i].dnum;
    }

    return out;
}

Dynamically loading libsomelib.so at runtime and calling add_data from C code is straightforward:

#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>

#include "somelib.h"

// Prototype for a function pointer for add_data
typedef DataPoint (*add_data_fn_t)(const DataPoint* dps, unsigned n);

int main(int argc, const char* argv[])
{
    void* libhandle = dlopen("./libsomelib.so", RTLD_LAZY);
    if (!libhandle) {
        fprintf(stderr, "dlopen error: %s\n", dlerror());
        exit(1);
    }

    printf("dlopen success: handle %p\n", libhandle);

    // We know the prototype of add_data so we can directly assign it to a
    // function pointer of the correct type.
    add_data_fn_t add_data_fn = dlsym(libhandle, "add_data");
    char* err = dlerror();
    if (err) {
        fprintf(stderr, "dlsym failed: %s\n", err);
        exit(1);
    }

    DataPoint dp[4] = {{2, 2.2}, {3, 3.3}, {4, 4.4}, {5, 5.5}};

    printf("Calling add_data\n");
    DataPoint dout = add_data_fn(dp, sizeof(dp) / sizeof(DataPoint));

    printf("dout = {%d, %lf}\n", dout.num, dout.dnum);
    return 0;
}

This works great. However, note a certain lack of flexibility. While the shared library can be discovered and loaded at runtime, the interface of the function we call from it has to be defined statically, at compile time - this is the function pointer prototype in the snippet above.

But what if we want the interface to be dynamic as well? In other words, what if we don't know until runtime what arguments the called function accepts? Alas, if standard C is all we have, we're stuck. The problem is that to call a function properly, the compiler has to know what arguments it accepts to translate the call to the proper machine code sequence according to the system's calling convention. When I disassemble both add_data and the call in main, I see this call sequence, which is in accordance with the System V AMD64 ABI [1]:

  • dps is passed in %rdi
  • n is passed in %esi
  • return value is in %xmm0

So to call a function whose signature is determined at runtime, we'd have to implement the calling convention ourselves, packing the arguments into registers and stack as appropriate and unpacking the return value. Moreover, this has to be implemented for each platform the code runs on. And it goes beyond saying that such code is not portable since standard C does not provide direct access to the stack or to the registers.

Luckily, a library exists that implements all of this for us.

libffi

libffi was designed to solve precisely the problem described above - provide a means to call a function from a shared object, while deciding at runtime which arguments the function accepts and which value it returns. Conceivably this can be useful for C code dynamically invoking other C code [2], but the main users of libffi are dynamic VM languages. Python uses libffi in its ctypes library, and other languages like Java, Ruby and Scheme use it in similar C FFI (Foreign Function Interface) libraries.

Without further ado, here's a version of the main program from above that uses libffi to call add_data from its shared library:

#include <dlfcn.h>
#include <ffi.h>
#include <stdio.h>
#include <stdlib.h>

#include "somelib.h"  // For the DataPoint type.

int main(int argc, const char* argv[])
{
    void* libhandle = dlopen("./libsomelib.so", RTLD_LAZY);
    if (!libhandle) {
        fprintf(stderr, "dlopen error: %s\n", dlerror());
        exit(1);
    }

    printf("dlopen success: handle %p\n", libhandle);

    // Assuming we don't know the prototype of add_data at compile-time, we
    // have to save the output of dlsym in a void* and then prepare the
    // calling sequence using libffi.
    void* add_data_fn = dlsym(libhandle, "add_data");
    char* err = dlerror();
    if (err) {
        fprintf(stderr, "dlsym failed: %s\n", err);
        exit(1);
    }

    // Describe the function arguments. Note that ffi_type_pointer is used
    // for any C pointer (the pointee type does not matter in the ABI).
    ffi_type* args[] = {&ffi_type_pointer, &ffi_type_uint};

    // Describe the DataPoint struct to libffi. Elements are described by a
    // NULL-terminated array of pointers to ffi_type.
    ffi_type* dp_elements[] = {&ffi_type_sint, &ffi_type_double, NULL};
    ffi_type dp_type = {.size = 0, .alignment = 0,
                        .type = FFI_TYPE_STRUCT, .elements = dp_elements};

    // Describe the interface of add_data to libffi.
    ffi_cif cif;
    ffi_status status = ffi_prep_cif(&cif, FFI_DEFAULT_ABI, 2, &dp_type,
                                     args);
    if (status != FFI_OK) {
        fprintf(stderr, "ffi_prep_cif failed: %d\n", status);
        exit(1);
    }

    // The avalues argument of ffi_call holds the addresses of arguments.
    // Since our first argument is a pointer itself, we can't just pass
    // &dp (since in C &array == array). So we create a pointer to dp and
    // pass its address.
    DataPoint dp[4] = {{2, 2.2}, {3, 3.3}, {4, 4.4}, {5, 5.5}};
    DataPoint* pdp = dp;
    unsigned nelems = sizeof(dp) / sizeof(DataPoint);
    void* values[] = {&pdp, &nelems};

    printf("Calling add_data via libffi\n");
    DataPoint dout;
    ffi_call(&cif, FFI_FN(add_data_fn), &dout, values);

    printf("dout = {%d, %lf}\n", dout.num, dout.dnum);
    return 0;
}

The code is heavily commented, so it should be easy to figure out what's going on. I just want to focus on a few interesting points:

  • The shared library is loaded as before. dlopen and dlsym are used. The result of dlsym is just placed in a void*, since we don't know the actual function pointer signature at compile time.
  • somelib.h is included just for the definition of the DataPoint type, since we want to actually pass data to add_data and get a result.
  • The signature of add_data is described dynamically, at runtime, by filling the ffi_cif data structure.

In terms of its implementation, libffi does as much as possible in portable C, but eventually has to resort to assembly routines written for each architecture and calling convention it supports. There routines perform the actual register and stack modifications around the call to the given function to make sure the call conforms to the calling convention. Note also that due to this extra work, calls via libffi are much slower than direct calls created by the compiler. In theory, it's possible to use JIT-ing to dynamically generate efficient calling code once the function signature is known, but AFAIK libffi does not implement this.

[1]I've compiled this example on my x64 Linux machine.
[2]I'm curious to hear about use cases, though. It seems to me that if you want to call code from C and don't even know the function signatures at compile time, other solutions (like serializing the arguments and return values, or some sort of message passing) is more commonplace.

For comments, please send me an email.