Plugins in C

Discovery & registration

The main application has a known directory in which it looks for plugin DSOs. In my implementation this directory's location is relative to the working directory, but it could be anywhere, really. It can also be specified in some kind of configuration file - many applications follow this route.

Once it knows the directory, the application goes over all files in it and looks for files that appear to be plugins - files ending with the .so extension, the convention for DSOs on Linux. It then tries to load these files with dlopen. Here's the relevant portion of the code [3]:

// Make sure the path to dlopen has a slash, for it to consider it
// an actual filesystem path and not just a lookup name.
dstring slashedpath = dstring_format("./%s", dstring_cstr(fullpath));

// Attempt to open the plugin DSO
void* libhandle = dlopen(dstring_cstr(slashedpath), RTLD_NOW);
dstring_free(slashedpath);
if (!libhandle) {
    printf("Error loading DSO: %s\n", dlerror());
    return NULL;
}

The story doesn't end here, however. To register itself with the application, a valid plugin is expected to have an initialization function which the application will call. The function's name must be init_<pluginname> where pluginname is the name of the plugin file without the .so extension. Take the tt.so plugin, for example. Its (non-static) initialization function must be named init_tt. This is the code that looks for the init function in the DSO:

// Attempt to find the init function and then call it
dstring initfunc_name = dstring_format("init_%s", dstring_cstr(name));
// dlsym returns void*, but we obviously need to cast it to a function
// pointer to be able to call it. Since void* and function pointers are
// mutually inconvertible in the eyes of C99, and -pedantic complains about
// a plain cast, we cast through a pointer-sized integer.
PluginInitFunc initfunc = (PluginInitFunc)
    (intptr_t) dlsym(libhandle, dstring_cstr(initfunc_name));
dstring_free(initfunc_name);
if (!initfunc) {
    printf("Error loading init function: %s\n", dlerror());
    dlclose(libhandle);
    return NULL;
}

The type PluginInitFunc is:

typedef int (*PluginInitFunc)(PluginManager*);

PluginManager is a central piece of the infrastructure; I will discuss it in more detail later. For now, it suffices to say that it is the interface between the application and plugins.

Anyhow, once the init function is successfully found in the plugin DSO, the application calls it, passing it a pointer to PluginManager. The init function is expected to return a non-negative value if everything is OK:

int rc = initfunc(pm);
if (rc < 0) {
    printf("Error: Plugin init function returned %d\n", rc);
    dlclose(libhandle);
    return NULL;
}

At this point, the plugin was discovered and has registered itself with the application - it was loaded from a shared library, and the initialization function was found and executed successfully.

All of the above is implemented in the plugin_discovery module (a pair of .h and .c files).

Application hooks

This is the place to discuss PluginManager. It's an object in the C sense of the word - the interface exposes an opaque data type and some functions that operate on it (it's all in plugin_manager.h/c).

PluginManager is used both by the application and by plugins. Plugins use it to register hooks. The application uses it to find all registered hooks and execute them. Similarly to the Python version of htmlize, there are two kinds of hooks - a hook for specific roles, and a hook for the whole contents. Here are the relevant callback function prototypes:

// Role hook. Will be called with: the role contents, DB and Post objects.
//
typedef dstring (*PluginRoleHook)(dstring, DB*, Post*);

// Contents hook. Will be called with: post contents, DB and Post objects.
//
typedef dstring (*PluginContentsHook)(dstring, DB*, Post*);

Note the DB and Post arguments - we'll discuss them later. These are the registration functions plugins can use to add hooks:

// Register a hook for a specific role.
// Note: rolename is copied to an internal data structure.
//
void PluginManager_register_role_hook(PluginManager* pm, dstring rolename,
                                      PluginRoleHook hook);

// Register a hook for contents.
//
void PluginManager_register_contents_hook(PluginManager* pm,
                                          PluginContentsHook hook);

This is the right time to show the full code of the tt.so plugin, which registers itself for the tt role, wrapping its contents in <tt>...</tt> tags:

static dstring tt_role_hook(dstring str, DB* db, Post* post) {
    return dstring_format("<tt>%s</tt>", dstring_cstr(str));
}


int init_tt(PluginManager* pm) {
    dstring rolename = dstring_new("tt");
    PluginManager_register_role_hook(pm, rolename, tt_role_hook);
    dstring_free(rolename);
    return 1;
}

The initialization function of the plugin (which, recall, must be called init_tt to be found) registers a role hook for the tt role with the plugin manager, and returns 1 for success. The hook itself is a simple function that performs the required transformation [4].

For completeness, this is the "application side" of the plugin manager API:

// Apply the registered role hooks to the given rolename/rolecontents,
// returning the string that should replace the role.
// The first plugin that agrees to handle this role is used. If no such plugin
// is found, NULL is returned.
//
dstring PluginManager_apply_role_hooks(PluginManager* pm,
                                       dstring rolename, dstring rolecontents,
                                       DB* db, Post* post);

// Apply the registered contents hooks to the given contents, returning
// the transformed contents.
// All registered hooks are composed:
//
//  while (has_plugins)
//      contents = apply_next_plugin(contents)
//
// If no contents plugin exists, NULL is returned.
//
dstring PluginManager_apply_contents_hooks(PluginManager* pm, dstring contents,
                                           DB* db, Post* post);

If you look into plugin_manager.c, you'll see that the implementation of these functions is pretty simple. PluginManager holds lists of registered hooks, and the PluginManager_apply_* functions simply walk these lists applying the hooks, when the application requests it.

Exposing application capabilities back to plugins

We've already seen an example of this above. The PluginManager API has a plugin-facing component for registering hooks, which technically is an application capability exposed to plugins. But there's more; I want to reimplement the same mock "database" API I used in the Python example, since it provides a very realistic example and is applicable in many situations.

At this point it's interesting to highlight an important difference between Python and C. In Python, due to duck typing, one module can just pass an object to another and the other module doesn't have to have any type information about this object - it can just call its methods. In C, things are not that easy. Therefore, to use the DB and Post objects, plugins need to include the application header file defining them (db.h). Note that due to the nature of dynamic linking on Linux, plugins don't have to actually link with the db.o object. More on this later.

To demonstrate this in code, here's a part of the narcissist.so plugin which turns all occurrences of "I" to "<b>I (username)</b>":

#include "db.h"

static dstring narcissist_contents_hook(dstring str, DB* db, Post* post) {
    dstring replacement = dstring_format("<b>I (%s)</b>",
                            dstring_cstr(Post_get_author(post)));
    ...
    ...
}

int init_narcissist(PluginManager* pm) {
    PluginManager_register_contents_hook(pm, narcissist_contents_hook);
    return 1;
}

All hooks get passed pointers to DB and Post objects. The plugin then uses the db.h API to access the Post object - in this case the Post_get_author function, which extracts the username from the post.

Exporting symbols from the application to plugins

Here are the compiler invocations required to build the tt.so plugin:

gcc -c plugins/tt.c -o plugins/tt.o -pedantic -g -Wall -std=c99 -fpic -I.
gcc -o plugins/tt.so plugins/tt.o -shared

This is a standard Linux DSO build: the sources are compiled with -fpic to generate position-independent code, and the DSO itself is built with -shared which tells the linker to create a shared library.

When creating a DSO with -shared, we don't have to link in object files that will be found in the application that loads the DSO. The plugin uses symbols from a number of object files - dstring.o, db.o, plugin_manager.o. Let's see how this looks in the symbol table:

$ readelf --dyn-syms plugins/narcissist.so

Symbol table '.dynsym' contains 23 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000798     0 SECTION LOCAL  DEFAULT    9
     2: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND dstring_len
     3: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND dstring_new_len
    <snip>
     8: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND Post_get_author
    <snip>

This is the dynamic symbol table, a section used by the dynamic linker on Linux for symbol management. Here it says that the symbols dstring_len, Post_get_author and others and undefined. The dynamic linker will expect to find them in the application loading the DSO. Otherwise, we'll get a symbol resolution error at runtime.

There's an important gotcha here. The linker will not export symbols from an application to plugins by default. It has to be explicitly told to do so by means of the --export-dynamic linker flag. Here's a portion of the ld manual page that describes this flag very well:

--export-dynamic
--no-export-dynamic
    When creating a dynamically linked executable, using the -E
    option or the --export-dynamic option causes the linker to add
    all symbols to the dynamic symbol table.  The dynamic symbol
    table is the set of symbols which are visible from dynamic
    objects at run time.

    If you do not use either of these options (or use the
    --no-export-dynamic option to restore the default behavior),
    the dynamic symbol table will normally contain only those
    symbols which are referenced by some dynamic object mentioned
    in the link.

    If you use "dlopen" to load a dynamic object which needs to
    refer back to the symbols defined by the program, rather
    than some other dynamic object, then you will probably need
    to use this option when linking the program itself.

    You can also use the dynamic list to control what symbols
    should be added to the dynamic symbol table if the output
    format supports it. See the description of --dynamic-list.

This behavior is easy to observe in our example, if you're interested. The main application htmlize_main is currently compiled with the --export-dynamic flag. If you look at its dynamic symbol table (readelf --dyn-syms), you'll see all global symbols exported. Recompile it without the flag, and you can check that the dynamic symbol table won't contain these symbols, and the dlopen call in plugin_discovery.c will fail with "undefined symbol" errors.

Symbol visibility between plugins

We've seen that special provision is required for the application's symbols to be visible inside pligins. The same is true about symbol visibility between plugins, though here the mechanism is different.

When the application loads a plugin with dlopen, the plugin's symbols can be found by calling dlsym in the application. However, what if other plugins need to use these symbols as well? By default, that won't work. To make it work, it's possible to pass the RTLD_GLOBAL flag to dlopen when opening the plugin we want to expose the symbols from. Symbols in this plugin DSO will be made available to resolve references in subsequently loaded DSOs.

Cross-DSO memory allocation

It's not hard to see in the htmlize example that some memory allocated in one DSO (or the main application), is released in another. Especially if you come from a Windows background, this may raise an eyebrow.

Cross-DSO memory allocation is most likely wrong when the C library is linked statically. This is because each DSO gets its own version of the C library, with its own book-keeping for malloc et al, so memory allocated in one DSO can't be released in another.

However, on Linux it's customary to link the C library dynamically. This is what happens by default, unless you explicitly request static linking. When linked dynamically, only a single version of the C library symbols exists in the process's address space during execution, and cross-DSO memory allocations and releases are safe.