Eli Bendersky's website - C & C++

Notes on the WASM Basic C ABI

2025-11-24T19:47:00-08:00

The WebAssembly/tool-conventions repository contains "Conventions supporting interoperability between tools working with WebAssembly".

Of special interest, in contains the Basic C ABI - an ABI for representing C programs in WASM. This ABI is followed by compilers like Clang with the wasm32 target. Rust is also switching to this ABI for extern "C" code.

This post contains some notes on this ABI, with annotated code samples and diagrams to help visualize what the emitted WASM code is doing. Hereafter, "the ABI" refers to this Basic C ABI.

Preface: the WASM value stack and linear memory

In these notes, annotated WASM snippets often contain descriptions of the state of the WASM value stack at a given point in time. Unless otherwise specified, "TOS" refers to "Top Of value Stack", and the notation [ x y ] means the stack has y on top, with x right under it (and possibly some other stuff that's not relevant to the discussion under x); in this notation, the stack grows "to the right".

The WASM value stack has no linear memory representation and cannot be addressed, so it's meaningless to discuss whether the stack grows towards lower or higher addresses. The value stack is simply an abstract stack, where values can be pushed onto or popped off its "top".

Whenever addressing is required, the ABI specifies explicitly managing a separate stack in linear memory. This stack is very similar to how stacks are managed in hardware assembly languages (except that in the ABI this stack pointer is held in a global variable, and is not a special register), and it's called the "linear stack".

Scalar parameters and returns

By "scalar" I mean basic C types like int, double or char. For these, using the WASM value stack is sufficient, since WASM functions can accept an arbitrary number of scalar parameters.

This C function:

int add_three(int x, int y, int z) {
  return x + y + z;
}

Will be compiled into something like:

(func $add_three (param i32 i32 i32) (result i32)
  local.get 1     ;; [ y ]
  local.get 0     ;; [ y  x ]
  i32.add         ;; [ x+y ]
  local.get 2     ;; [ x+y  z ]
  i32.add         ;; [ x+y+z ]
)

And can be called by pushing three values onto the stack and invoking call $add_three.

The ABI specifies that all integral types 32-bit and smaller will be passed as i32, with the smaller types appropriately sign or zero extended. For example, consider this C function:

char add_three_chars(char x, char y, char z) {
  return x + y + z;
}

It's compiled to the almost same code as add_three:

(func $add_three_chars (param i32 i32 i32) (result i32)
  local.get 1
  local.get 0
  i32.add
  local.get 2
  i32.add
  i32.extend8_s
)

Except the last i32.extend8_s, which takes the lowest 8 bits of the value on TOS and sign-extends them to the full i32 (effectively ignoring all the higher bits). Similarly, when $add_three_chars is called, each of its parameters goes through i32.extend8_s.

There are additional oddities that we won't get deep into, like passing __int128 values via two i64 parameters.

Pointers

C pointers are just scalars, but it's still educational to review how they are handled in the ABI. Pointers to any type are passed in i32 values; the compiler knows they are pointers, though, and emits the appropriate instructions. For example:

int add_indirect(int x, int y, int* sum) {
    *sum = x + y;
    return *sum;
}

Is compiled to:

(func $add_indirect (param i32 i32 i32) (result i32)
  local.get 2         ;; [ ptr_sum ]
  local.get 1         ;; [ ptr_sum  y ]
  local.get 0         ;; [ ptr_sum  y  x ]
  i32.add             ;; [ ptr_sum  x+y ]
  local.tee 0         ;; x <- x+y, leaving stack intact
  i32.store           ;; store x+y into *ptr_sum
  local.get 0         ;; [ x+y ] for returning
)

Recall that in WASM, there's no difference between an i32 representing an address in linear memory and an i32 representing just a number. i32.store expects [ addr value ] on TOS, and does *addr = value.

Note that the x parameter isn't needed any longer after the sum is computed, so it's reused later on to hold the return value. WASM parameters are treated just like other locals (as in C).

Passing parameters through linear memory

According to the ABI, while scalars and single-element structs or unions are passed to a callee via WASM function parameters (as shown above), for larger aggregates the compiler utilizes linear memory.

Specifically, each function gets a "frame" in a region of linear memory allocated for the linear stack. This region grows downwards from high to low addresses [1], and the global $__stack_pointer points at the bottom of the frame:

Consider this code:

struct Pair {
    unsigned x;
    unsigned y;
};

__attribute__((noinline))
unsigned pair_calculate(struct Pair pair) {
    return 7*pair.x + 3*pair.y;
}

unsigned do_work(unsigned x, unsigned y){
    struct Pair pp = {.x = x, .y = y};
    return pair_calculate(pp);
}

When do_work is compiled to WASM, prior to calling pair_calculate it copies pp into a location in linear memory, and passes the address of this location to pair_calculate. This location is on the linear stack, which is maintained using the $__stack_pointer global. Here's the compiled WASM for do_work (I also gave its local variable a meaningful name, for readability):

(func $do_work (param i32 i32) (result i32)
  (local $sp i32)
  global.get $__stack_pointer
  i32.const 16
  i32.sub
  local.tee 2                   ;; sp <- __stack_pointer - 16
  global.set $__stack_pointer   ;; update __stack_pointer as well
  local.get 2
  local.get 1
  i32.store offset=12           ;; mem[sp+12] = y
  local.get 2
  local.get 0
  i32.store offset=8            ;; mem[sp+8] = x
  local.get 2                   ;; [ sp ]
  local.get 2                   ;; [ sp  sp ]

  ;; Do a 64-bit load from mem[sp+8], this loads the entire pair into
  ;; a single i64.
  i64.load offset=8 align=4
  i64.store                     ;; mem[sp] = pair-as-i64
  local.get 2                   ;; [ sp ]
  call $pair_calculate          ;; call pair_calculate, passing it sp
  local.set 1                   ;; x = result
  local.get 2                   ;; [ sp ]
  i32.const 16
  i32.add                       ;; [ sp+16 ]
  global.set $__stack_pointer   ;; __stack_pointer back to its original value
  local.get 1                   ;; [ x ] for return
)

Some notes about this code:

There are two instance of the pair pp in linear memory prior to the call to pair_calculate: the original one from the initialization statement (at offset 8), and a copy created for passing into pair_calculate (at offset 0). Theoretically, as pp is unused used after the call, the compiler could do better here and keep only a single copy.
The stack pointer is decremented by 16, and restored at the end of the function.
The first few instructions - where the stack pointer is adjusted - are usually called the prologue of the function. In the same vein, the last few instructions where the stack pointer is reset back to where it was at the entry are called the epilogue.

Before pair_calculate is called, the linear stack looks like this:

Following the ABI, the code emitted for pair_calculate takes Pair* (by reference, instead of by value as the original C code):

(func $pair_calculate (param i32) (result i32)
  local.get 0           ;; [ addr ]

  ;; Recall that what's stored at mem[addr] is a pair, with 'x', followed by 'y'.
  i32.load offset=4     ;; [ pair.y ]
  i32.const 3
  i32.mul               ;; [ 3*pair.y ]
  local.get 0
  i32.load              ;; [ 3*pair.y  pair.x ]
  i32.const 7
  i32.mul               ;; [ 3*pair.y  7*pair.x ]
  i32.add               ;; [ 3*pair.y+7*pair.x ]
)

Each function that needs linear stack space is responsible for adjusting the stack pointer and restoring it to its original place at the end. This naturally enables nested function calls; suppose we have some function a calling function b which, in turn, calls function c, and let's assume all of these need to allocate space on the linear stack. This is how the linear stack looks after c's prologue:

Since each function knows how much stack space it has allocated, it's able to properly restore $__stack_pointer to the bottom of its caller's frame before returning.

Returning values through linear memory

What about returning values of aggregate types? According to the ABI, these are also handled indirectly; a pointer parameter is prepended to the parameter list of the function. The function writes its return value into this address.

The following function:

struct Pair make_pair(unsigned x, unsigned y) {
  struct Pair pp = {.x = x, .y = y};
  return pp;
}

Is compiled to:

;; Note that the WASM function has three parameters an no return values.
;; The first parameter (local 0) is the address where it should store its
;; return value.
(func $make_pair (param i32 i32 i32)
  local.get 0
  local.get 2
  i32.store offset=4      ;; ret_addr[4] = y
  local.get 0
  local.get 1
  i32.store               ;; ret_addr[0] = x
)

Here's a function that calls it:

unsigned do_work(unsigned x, unsigned y) {
  struct Pair pp = make_pair(x, y);
  return 7*pp.x + 3*pp.y;
}

And the corresponding WASM:

(func $do_work (param i32 i32) (result i32)
  ;; local 2 to hold the address for make_pair's return value.
  (local i32)
  global.get $__stack_pointer   ;; sp <- __stack_pointer - 16
  i32.const 16
  i32.sub
  local.tee 2                   ;; save sp in local 2
  global.set $__stack_pointer
  local.get 2
  i32.const 8
  i32.add                       ;; [ sp+8 ]
  local.get 0                   ;; [ sp+8  x ]
  local.get 1                   ;; [ sp+8  x  y ]

  ;; make_pair is called with three parameters: the address for where to
  ;; store its return pair, x and y.
  call $make_pair
  local.get 2
  i32.load offset=8
  local.set 1                   ;; local 1 = pp.x
  local.get 2
  i32.load offset=12
  local.set 0                   ;; local 0 = pp.y
  local.get 2
  i32.const 16
  i32.add
  global.set $__stack_pointer   ;; __stack_pointer back to its original value
  local.get 0
  i32.const 3
  i32.mul                       ;; [ 3*pp.y ]
  local.get 1
  i32.const 7
  i32.mul                       ;; [ 3*pp.y  7*pp.x ]
  i32.add                       ;; [ 3*pp.y+7*pp.x ]
)

Note that this function only uses 8 bytes of its stack frame, but allocates 16; this is because the ABI dictates 16-byte alignment for the stack pointer.

Advanced topics

There are some advanced topics mentioned in the ABI that these notes don't cover (at least for now), but I'll mention them here for completeness:

"Red zone" - leaf functions have access to 128 bytes of red zone below the stack pointer. I found this difficult to observe in practice [2]. Since we don't issue system calls directly in WASM, it's tricky to conjure a realistic leaf function that requires the linear stack (instead of just using WASM locals).
A separate frame pointer (global value) to be used for functions that require dynamic stack allocation (such as using C's VLAs).
A separate base pointer to be used for functions that require alignment > 16 bytes on the stack.

[1]

This is similar to x86. For the WASM C ABI, a good reason is provided for the direction: WASM load and store instructions have an unsigned constant called offset that can be used to add a positive offset to the address parameter without extra instructions.

Since $__stack_pointer points to the lowest address in the frame, these offsets can be used to efficiently access any value on the stack.

[2]	If you have a nice example showing it using Clang/LLVM, please drop me a note!

Implementing Forth in Go and C

2025-08-26T20:38:00-07:00

I first ran into Forth about 20 years ago when reading a book about designing embedded hardware. The reason I got the book back then was to actually learn more about the HW aspects, so having skimmed the Forth chapter I just registered an "oh, this is neat" mental note and moved on with my life. Over the last two decades I heard about Forth a few more times here and there, such as that time when Factor was talked about for a brief period, maybe 10-12 years ago or so.

It always occupied a slot in the "weird language" category inside my brain, and I never paid it much attention. Until June this year, when a couple of factors combined fortuitously:

After spending much of the earlier part of 2025 exploring the inner workings of LLMs and digging in random mathy and algorithmic topics, I had an itch to just write some code.
I somehow found Dave Gauer's page about Forth and also the one on Implementing a Forth.

And something clicked. I'm going to implement a Forth, because... why not?

So I spent much of my free hacking time over the past two months learning about Forth and implementing two of them.

Forth: the user level and the hacker level

It's useful to think of Forth (at least standard Forth, not offshoots like Factor) as having two different "levels":

User level: you just want to use the language to write programs. Maybe you're indeed bringing up new hardware, and find Forth a useful calculator + REPL + script language. You don't care about Forth's implementation or its soul, you just want to complete your task.
Hacker level: you're interested in the deeper soul of Forth. Isn't it amazing that even control flow constructs like IF...THEN or loops like BEGIN...UNTIL are just Forth words, and if you wanted, you could implement your own control flow constructs and have them be first-class citizens, as seamless and efficient as the standard ones?

Another way to look at it (useful if you belong to a certain crowd) is that user-level Forth is like Lisp without macros, and hacker-level Forth has macros enabled. Lisp can still be great and useful without macros, but macros take it to an entire new level and also unlock the deeper soul of the language.

This distinction will be important when discussing my Forth implementations below.

goforth and ctil

There's a certain way Forth is supposed to be implemented; this is how it was originally designed, and if you get closer to the hacker level, it becomes apparent that you're pretty much required to implement it this way - otherwise supporting all of the language's standard words will be very difficult. I'm talking about the classical approach of a linked dictionary, where a word is represented as a "threaded" list [1], and this dictionary is available for user code to augment and modify. Thus, much of the Forth implementation can be written in Forth itself.

The first implementation I tried is stubbornly different. Can we just make a pure interpreter? This is what goforth is trying to explore (the Go implementation located in the root directory of that repository). Many built-in words are supported - definitely enough to write useful programs - and compilation (the definition of new Forth words using : word ... ;) is implemented by storing the actual string following the word name in the dictionary, so it can be interpreted when the word is invoked.

This was an interesting approach and in some sense, it "works". For the user level of Forth, this is perfectly usable (albeit slow). However, it's insufficient for the hacker level, because the host language interpreter (the one in Go) has all the control, so it's impossible to implement IF...THEN in Forth, for example (it has to be implemented in the host language).

That was a fun way to get a deeper sense of what Forth is about, but I did want to implement the hacker level as well, so the second implementation - ctil - does just that. It's inspired by the jonesforth assembly implementation, but done in C instead [2].

ctil actually lets us implement major parts of Forth in Forth itself. For example, variable:

: variable create 1 cells allot ;

Conditionals:

\ IF, ELSE, THEN work together to compile to lower-level branches.
\
\ IF ... THEN compiles to:
\   0BRANCH OFFSET true-part rest
\ where OFFSET is the offset of rest
\
\ IF ... ELSE ... THEN compiles to :
\   0BRANCH OFFSET true-part BRANCH OFFSET2 false-part rest
\ where OFFSET is the offset of false-part and OFFSET2 is the offset of rest
: if immediate
  ' 0branch ,
  here
  0 ,
  ;

: then immediate
  dup
  here swap -
  swap
  ! ;

: else immediate
  ' branch ,
  here
  0 ,
  swap
  dup
  here swap -
  swap
  ! ;

These are actual examples of ctil's "prelude" - a Forth file loaded before any user code. If you understand Forth, this code is actually rather mind-blowing. We compile IF and the other words by directly laying our their low-level representation in memory, and different words communicate with each other using the data stack during compilation.

Thoughts on Forth itself

Forth made perfect sense in the historic context in which it was created in the early 1970s. Imagine having some HW connected to your computer (a telescope in the case of Forth's creator), and you have to interact with it. In terms of languages at your disposal - you don't have much, even BASIC wasn't invented yet. Perhaps your machine still didn't have a C compiler ported to it; C compilers aren't simple, and C isn't very great for exploratory scripting anyway. So you mostly just have your assembly language and whatever you build on top.

Forth is easy to implement in assembly and it gives you a much higher-level language; you can use it as a calculator, as a REPL, and as a DSL for pretty much anything due to its composable nature.

Forth certainly has interesting aspects; it's a concatenative language, and thus inherently point-free. A classical example is that instead of writing the following in a more traditional syntax:

eat(bake(prove(mix(ingredients))))

You just write this:

ingredients mix prove bake eat

There is no need to explicitly pass parameters, or to explicitly return results. Everything happens implicitly on the stack.

This is useful for REPL-style programming where you use your language not necessarily for writing large programs, but more for interactive instructions to various HW devices. This dearth of syntax is also what makes Forth simple to implement.

All that said, in my mind Forth is firmly in the "weird language" category; it's instructive to learn and to implement, but I wouldn't actually use it for anything real these days. The stack-based programming model is cool for very terse point-free programs, but it's not particularly readable and hard to reason about without extensive comments, in my experience.

Consider the implementation of a pretty standard Forth word: +!. It expects and address at the top of stack, and an addend below it. It adds the addend to the value stored at that address. Here's a Forth implementation from ctil's prelude:

: +!        ( addend addr -- )
  tuck      ( addr addend addr )
  @         ( addr addend value-at-addr )
  +         ( addr updated-value )
  swap      ( updated-value addr )
  ! ;

Look at that stack wrangling! It's really hard to follow what goes where without the detailed comments showing the stack layout on the right of each instruction (a common practice for Forth programs). Sure, we can create additional words that would make this simpler, but that just increases the lexicon of words to know.

My point is, there's fundamental difficulty here. When you see this C code:

int func(int a, int b) {
  return foo(a, bar(b));
}

Even without any documentation, you can immediately know several important things:

bar has one parameter and one return value
foo has two parameters and one return value
func also has two parameters and one return value
It's immediately obvious how the various values flow from one function call to the next.

Written in Forth [3]:

: func bar foo ;

How can you know the arity of the functions without adding explicit comments? Sure, if you have a handful of words like bar and foo you know like the back of your hand, this is easy. But imagine reading a large, unfamiliar code base full of code like this and trying to comprehend it.

Summary and links

The source code of my goforth project is on GitHub; both implementations are there, with a comprehensive test harness that tests both.

The learn Forth itself, I found these resources very useful:

Dave Gauer's Forth page
Starting Forth - a free online book / tutorial on Forth for beginners

To learn how to implement Forth:

Dave Gauer's page on Implementing a Forth
jonesforth implementation
Threaded Interpretive Languages - an old but nice book that explains how Forth implementations typically work

Implementing Forth is a great self-improvement project for a coder; there's a pleasantly challenging hump of understanding to overcome, and you gain valuable insights into stack machines, interpretation vs. compilation and mixing these levels of abstraction in cool ways.

Also, implementing programming languages from scratch is fun! It's hard to beat the feeling of getting to interact with your implementation for the first time, and then iterating on improving it and making it more featureful. Just one more word!

[1]	This has nothing to do with threads in the sense of concurrency. Rather, it's thread like in sewing, where the elements of the list are all connected to each other as if with a thread. See this page for more details.

[2]

Which is another deviation from the norm. Forth is really supposed to be implemented in assembly - this is what it was designed for, and it's very clear from its structure that it must be so in order to achieve peak performance.

But where's the fun in doing things the way they were supposed to be done? Besides, jonesforth is already a perfectly fine Forth implementation in assembly, so I wouldn't have learned much by just copying it.

I had a lot of fun coding in C for this one; it's been a while since I last wrote non-trivial amounts of C, and I found it very enjoyable.

[3]	Assuming the convention that multi-parameter functions have their parameters pushed to the stack from left to right.

Plugins case study: Envoy WASM extensions

2023-06-03T15:14:00-07:00

Envoy is a configurable proxy that serves a prominent role in modern cloud-native projects; for example, it's used in many k8s deployments to provide inter-service communication (through Istio). In this post, I'd like to describe Envoy's extension mechanism as a case study of using WebAssembly for plugins.

Some background

Imagine a complex service-based infrastructure, in which service A has to communicate with service B. This is typically done via HTTP/REST or some RPC mechanism, but there are a lot of complex networking details to deal with: service discovery (I just want to send a message to an instance of service B, but which address/port is it on?), load balancing, retries, etc. Instead of having each service deal with this complexity, we can set up Envoy to run as a sidecar, and ask it to handle all of it. Then the actual services can focus on our business logic.

Here's a handy diagram from https://istio.io that demonstrates this (Envoy is the Proxy boxes):

System diagram of envoy proxies communicating for microservices

As expected - for such a sophisticated piece of software - Envoy users frequently need to customize it in various ways for their projects. For example, we may want to define custom filters; this is a kind of middleware.

Envoy's original approach to extensions was to support writing C++ to link custom filters with Envoy itself. This, of course, is awkward for many reasons - such as having to distribute your own Envoy binaries instead of using the standard ones. Also, the filter API was not really designed to be stable so keeping up with changes was an issue; and finally, few people like writing C++ these days.

So the Envoy team came up with an alternative approach: Lua extensions.

The Lua programming language was designed for extensions and plugins; it's a small and simple language, and its implementation is also small and simple - making it easy to embed. You can write some Lua code either directly in your configuration file or a separate file it points to, and there's an API exposed to Lua that the extension can interact with.

The Lua extension method is fully supported in Envoy and is currently in a stable state, but some folks weren't too keen on learning yet another programming language just for the sake of writing filters for their proxy. Lua is not particularly prominent in the Cloud world (which is mostly dominated by Go, Python, Java and some other languages). Therefore, the Envoy maintainers have created yet another way to extend it [1] - with WebAssembly.

WASM extensions

WASM extensions are still experimental in Envoy at the time of writing, but it's an intriguing approach and the main subject of this post. WASM elegantly solves the problems of the other extension methods as follows:

The WASM extension is compiled into a .wasm file that the Envoy config can point at, and is loaded dynamically at runtime. It doesn't require recompiling and distributing a custom version of Envoy.
The extension can use any programming language that compiles down to WASM, and that covers a lot of languages these days. Your entire service infrastructure is written in Go and you don't want to wrangle C++ or learn Lua just for the proxy filters? No problem - Go compiles to WASM and there's even an SDK to help writing Envoy filters in it.

To this end, Envoy embeds v8 as a WASM VM. All that remains is to define the interface between these WASM extension modules and Envoy itself.

The Proxy-Wasm ABI

WebAssembly itself defines:

A bytecode format (with an equivalent text format) and its execution semantics
A way for WASM modules to export functions and data to the host environment
A way for WASM modules to import functions and data from the host environment

And that's about it. Everything else is left to the specific system implementer to figure out. Moreover, the data types WASM supports are very limited - essentially fixed-width integers and floats; users are expected to build their own higher-level data structures on top of these using addresses into WASM's linear heap memory, if needed.

In a previous post I've talked about WASI - an API and ABI that enables OS-like functionality in WASM code. While WASI is useful for exposing WASM modules to the outside world in a vetted way, it's somewhat limited for complex host-wasm interactions, because at the moment the only way for this to happen is via interfaces like stdin/stdout [2].

Therefore, systems that require sophisticated interactions between the host and WASM extensions are left to define their own interfaces. Which is exactly what the Envoy developers ended up creating: the Proxy-Wasm ABI [3].

The ABI is fairly low level, and it has two parts. One is Functions implemented in the WASM module. These are functions exported from WASM (the custom extension) and imported by the host (Envoy or another proxy). For example, proxy_on_request_headers is exported by the WASM module as a callback to handle headers for HTTP requests sailing through the proxy.

This is the signature of proxy_on_request_headers:

params:
    i32 (uint32_t) context_id
    i32 (size_t) num_headers
    i32 (bool) end_of_stream
returns:
    i32 (proxy_action_t) next_action

The import is done in the proxy-wasm-cpp-host project which is a dependency of Envoy. This project implements the host side of Proxy-wasm for C++ hosts.

What should the extension do within proxy_on_request_headers, though? It can do things like ask Envoy about the actual HTTP headers it sees with proxy_get_header_map_value. This is in the second part of the ABI, Functions implemented in the host environment. Its signature is:

params:
    i32 (proxy_map_type_t) map_type
    i32 (const char*) key_data
    i32 (size_t) key_size
    i32 (const char**) return_value_data
    i32 (size_t*) return_value_size
returns:
    i32 (proxy_result_t) call_result

As you can see this is a very low level ABI; all parameters are either pointers (addresses in WASM's linear memory) or constants of predefined types. Since WASM severely restricts the types of function parameters and return values, and both the WASM module and the host can be implemented in very diverse programming languages, there's not much choice here. Writing the glue code on the WASM-host interface is tedious and low-level.

This is where the high-level SDKs come in.

The Go SDK for Proxy-wasm

Suppose we're writing our Envoy extension module in Go (a reasonable choice given the dominance of Go in the Cloud Native / k8s / Istio ecosystem). It seems like hooking up a simple extension to snoop on all the HTTP traffic going through the proxy and logging the HTTP headers is quite a bit of work.

Luckily, the good folks at Tetrate created the Go SDK for Proxy-Wasm. This SDK handles all the Proxy-Wasm ABI mechanics and presents a clean, pure Go API to extension writers that won't have to worry about low level WASM details.

Here's how the task of "snoop on HTTP traffic and log headers" looks using the Go SDK:

func (ctx *httpHeaders) OnHttpRequestHeaders(numHeaders int, endOfStream bool) types.Action {
  hs, err := proxywasm.GetHttpRequestHeaders()
  if err != nil {
    proxywasm.LogCriticalf("failed to get request headers: %v", err)
  }

  for _, h := range hs {
    proxywasm.LogInfof("request header --> %s: %s", h[0], h[1])
  }
  return types.ActionContinue
}

Let's explore how both sides of the ABI (host-implemented and module-implemented) are handled by the Go SDK. Starting with the WASM-calls-host side, this is proxywasm.GetHttpRequestHeaders:

func GetHttpRequestHeaders() ([][2]string, error) {
  return getMap(internal.MapTypeHttpRequestHeaders)
}

It's just a wrapper around a more general getMap function with a map type that the ABI defines. The return type is a slice of 2-element arrays (key/value).

func getMap(mapType internal.MapType) ([][2]string, error) {
  var rvs int
  var raw *byte

  st := internal.ProxyGetHeaderMapPairs(mapType, &raw, &rvs)
  if st != internal.StatusOK {
    return nil, internal.StatusToError(st)
  } else if raw == nil {
    return nil, types.ErrorStatusNotFound
  }

  bs := internal.RawBytePtrToByteSlice(raw, rvs)
  return internal.DeserializeMap(bs), nil
}

internal.ProxyGetHeaderMapPairs is actually an ABI-defined function that's imported from the host (as proxy_get_header_map_pairs). It writes raw pointers to its output parameters, so the rest of getMap deals with converting those into Go data types.

On the host side, proxy_get_header_map_pairs is mapped to a C++ function in this file.

Now the host-calls-WASM side. The Go SDK has the following function:

//export proxy_on_request_headers
func proxyOnRequestHeaders(contextID uint32, numHeaders int, endOfStream bool) types.Action {
  if recordTiming {
    defer logTiming("proxyOnRequestHeaders", time.Now())
  }
  ctx, ok := currentState.httpContexts[contextID]
  if !ok {
    panic("invalid context on proxy_on_request_headers")
  }

  currentState.setActiveContextID(contextID)
  return ctx.OnHttpRequestHeaders(numHeaders, endOfStream)
}

Note the //export annotation that tells the compiler to export this function from the WASM module. To be clear, the entire SDK - along with our custom code - gets compiled into a .wasm file that the host loads, and the //export tag makes the Go compiler place this function in the WASM function export table that the host has access to.

Once the host invokes it, it calls the OnHttpRequestHeaders method on the context, which is user-defined as shown above. Hopefully this example gives a taste of what the SDK does for us - it provides a higher-level, language-idiomatic API on top of a low-level, language-agnostic ABI.

The Go SDK is just an example; there are other SDKs that exist for developing WASM extensions for Envoy - for example in Rust or in C++.

One small wrinkle in this story is that the Go SDK only supports the TinyGo compiler at this time, not the default Go toolchain. This is because the default toolchain doesn't have sufficient WASM support yet, but this situation is changing; in Go 1.21 it has added WASI support and work is ongoing on additional features that should make it possible to develop Envoy extensions using the standard toolchain.

Fundamental plugin concepts in this case study

Let's see how this case study of Envoy extensions with WASM measures against the Fundamental plugin concepts that were covered several times on this blog.

Discovery

Envoy "discovers" available extensions trivially, because they have to be explicitly specified in its configuration file. The config file lists the extensions and where to find them; for WASM, this could be either a local .wasm file or a URL pointing to a file stored remotely (e.g. some cloud storage bucket).

Registration

The WASM extension registers functionality with Envoy by exporting certain functions from the WASM module. When Envoy loads an extension, it scans the list of exported functions for known names. For example, if the extension exports proxy_on_request_headers, Envoy will call it for HTTP headers. If the extension doesn't export such a function, Envoy will assume it's not interested in this particular callback.

Another interesting example of how this functionality is used is the proxy_abi_version_X_Y_Z function. An extension will export this function with an actual ABI version replacing X, Y an Z. Envoy will look for a function with the proxy_abi_version_* prefix, and from its name will determine which version of the ABI the WASM module was compiled against.

Hooks

This is mostly covered in the previous section. There are multiple callbacks a WASM extension can register by exporting them from the WASM module; proxy_on_request_headers is one example out of many defined in the ABI.

Exposing an application API to plugins

This is the Functions implemented in the host environment part of the Proxy-Wasm ABI; we've seen an example of one of them - proxy_get_header_map_pairs. The ABI defines others, like proxy_log for emitting log messages to Envoy's log. These functions let extensions call into Envoy.

Conclusion

As you can see from the string of posts this year, I'm pretty excited about the non-browser uses of WASM, particularly in the area of plugins. The FAAS post presented one interesting possibility - using the current (limited but functional) WASI for the host/plugin interface.

What this post shows is a case study of a much more advanced extension system; the capabilities and performance requirements of custom network filter plugins are just way beyond what WASI can provide, so the Envoy developers ended up creating their own ABI. It's fascinating to study how such an ABI affects plugin development and what kind of ecosystem it spawns.

[1]

Note that I'm not trying to criticize the existing extension mechanisms in Envoy in any way. Both work, and are used to solve real business problems. As a project like Envoy grows in popularity and usage, it's inevitable that it will spawn more options for different people to accomplish their tasks with it. Such is the way of software.

[2]	The WASI folks are working on extensions to allow sockets and also more complex data to be shared between WASM and hosts in an RPC-like manner; this may enable greatly improved wasm-host interfaces in the future.

[3]

This all sounds great - the way things should be - until reality kicks in. While doing research for this post I discovered that the Proxy-wasm ABI, while clearly and carefully specified, is in-fact out of date and the "real" definition lives within the Envoy source code. It's yet another case of "the ABI is whatever its main implementation does", even though other proxies implement it already (MOSN for example).

This is especially often the case in my favorite domain - systems programming. Sigh, such is life. The rest of the post talks about the de-facto specification, relying on the Envoy source code more than the written down ABI. Hopefully at some future point the ABI is updated and I can rewrite this footnote.

A shout out to Adrian Cole and Takeshi Yoneda for confirming these findings, and the useful chats about all things WASM, WASI and Go in general.

"Beating" C with 400 lines of unoptimized assembly

2019-11-23T09:46:00-08:00

Earlier this week I ran into a fun quick blog post named Beating C with 70 lines of Go, which reimplements the basic functionality of wc in Go using various approaches and compares their performance. Apparently it's inspired by an earlier Haskell-based post and several other offshoots.

This reminded me of my earlier post about reimplementing wc in pure x64 assembly, where I also measured the performance of my program against wc.

The optimized approach taken in the Go implementation is very similar to what I did in assembly, so it seemed like an interesting comparison. I started by generating a ~580 MiB file using xmlgen and ran the various versions against each other:

LC_TYPE=POSIX wc: 2.13 sec
wc-naive.go: 3.53 sec
wc-chunks.go: 1.37 sec
wcx64: 1.2 sec

Note the LC_TYPE setting for the system's wc. This is important for a fair comparison, because without this wc will attempt to do utf-8 decoding on all bytes in the file, which results in significant slowdowns. Since the Go versions use byte-counts and so does my wcx64, I force the comparison to be fair. In fact, this isn't a bad result for Go - the straightforward solution is almost as fast as the same approach direct-coded in assembly!

The Go blog post follows with parallelized versions which are much faster than the serial one, but I'm excluding it here because all the other competitors are single-threaded. This is not a serious benchmark anyway. If you prefer to be serious, this response using SIMD-optimized C blows everything out of the water:

fastlwc: 0.11 sec

The conclusion? Well, there's no real conclusion here, beyond that coding exercises like this are fun in any language :-)

Faster XML stream processing in Go

2019-07-22T05:37:00-07:00

XML processing was all the rage 15 years ago; while it's less prominent these days, it's still an important task in some application domains. In this post I'm going to compare the speed of stream-processing huge XML files in Go, Python and C and finish up with a new, minimal module that uses C to accelerate this task for Go. All the code shown throughout this post is available in this GitHub repository the new Go module is here.

What does XML stream processing mean?

First, let's define the problem at hand in more detail. Roughly speaking, there are two ways we can process data from a file:

Read the whole file into memory at once, and then proces the data in memory.
Read the file in chunks, process each chuck, without having the whole data in memory at any given time.

In many ways, (1) is more convenient because we can easily go back to any part of the file. However, in some situations (2) is essential; specifically, when the file is very large. This is where stream processing comes in. If our input file is 500 GiB, we're unlikely to be able to read it into memory and have to process it in parts. Even for smaller files that would theoretically fit into RAM, it's not always a good idea to read them wholly; this dramatically increases the active heap size and can cause performance issues in garbage-collected languages.

The task

For this benchmark, I'm using xmlgen to create a 230 MiB XML file [1]. A tiny fragment of the file may look like this:

<?xml version="1.0" standalone="yes"?>
<site>
  <regions>
    <asia>
      <item id="item0">
        <location>United States</location>
        <quantity>1</quantity>
        <name>duteous nine eighteen </name>
        <payment>Creditcard</payment>
        ...
      </item>
    </asia>
  </regions>
</site>

The task is to find how many times "Africa" appears in the data of the <location> tag throughout the document.

Baseline - using the Go standard library

Let's start with a baseline implementation - using the standard library's encoding/xml package. While the package's Unmarshal mode will parse the whole file in one go, it can also be used to process XML token by token and selectively parse interesting elements. Here is the code:

package main

import (
  "encoding/xml"
  "fmt"
  "io"
  "log"
  "os"
  "strings"
)

type location struct {
  Data string `xml:",chardata"`
}

func main() {
  f, err := os.Open(os.Args[1])
  if err != nil {
    log.Fatal(err)
  }
  defer f.Close()

  d := xml.NewDecoder(f)
  count := 0
  for {
    tok, err := d.Token()
    if tok == nil || err == io.EOF {
      // EOF means we're done.
      break
    } else if err != nil {
      log.Fatalf("Error decoding token: %s", err)
    }

    switch ty := tok.(type) {
    case xml.StartElement:
      if ty.Name.Local == "location" {
        // If this is a start element named "location", parse this element
        // fully.
        var loc location
        if err = d.DecodeElement(&loc, &ty); err != nil {
          log.Fatalf("Error decoding item: %s", err)
        }
        if strings.Contains(loc.Data, "Africa") {
          count++
        }
      }
    default:
    }
  }

  fmt.Println("count =", count)
}

I made sure to double check that the memory usage of this program stays bounded and low while processing a large file - the maximum RSS was under 7 MiB while processing our 230 MiB input file. I'm verifying this for all the programs presented in this post using /usr/bin/time -v on Linux.

This program takes 6.24 seconds to process the whole file and print out the result.

Python implementation

The first Python implementation uses the xml.etree.ElementTree module from the standard library:

import sys
import xml.etree.ElementTree as ET

count = 0
for event, elem in ET.iterparse(sys.argv[1], events=("end",)):
    if event == "end":
        if elem.tag == 'location' and elem.text and 'Africa' in elem.text:
            count += 1
        elem.clear()

print('count =', count)

The key here is the elem.clear() call. It ensures that each element gets discarded afer parsing it fully, so the memory usage won't grow linearly with the size of the file (unless the file is pathological). This program takes 3.7 seconds to process the whole file - much faster than our Go program. Why is that?

While the Go program uses 100% Go code for the task (encoding/xml is implemented entirely in Go), the Python program is using a C extension (most of ElementTree is written in C) wrapping a fast XML parser in C - libexpat. The bulk of the work here is done in C, which is faster than Go. The performance of encoding/xml is further discussed in this issue, though it's an old one and the performance has been somewhat optimized since then.

An alternative XML parsing library for Python is lxml, which uses libxml underneath. Here's a Python version using lxml:

import sys
from lxml import etree

count = 0
for event, elem in etree.iterparse(sys.argv[1], events=("end",)):
    if event == "end":
        if elem.tag == 'location' and elem.text and 'Africa' in elem.text:
            count += 1
        elem.clear()

print('count =', count)

This looks very similar to the previous version, and that's on purpose. lxml has an etree-compatible API to make transition from the standard library smoother. This version also takes around 3.7 seconds for our 230 MiB file.

The reason I'm including lxml here is that it will run faster than xml.etree.ElementTree when slurping the whole file, for our particular file size. I want to highlight that this is outside of the scope for my experiment, because I only care about streaming processing. The only way (that I'm aware of!) to successfully process a 500 GiB file with lxml would be by using iterparse.

How fast can it run?

Based on the measurements presented here, Go is about 68% slower than Python for parsing a large XML file in a streaming fashion. While Go usually compiles to a much faster code than pure Python, the Python implementations have the backing of efficient C libraries with which it's difficult to compete. I was curious to know how fast it could be, in theory [2].

To answer this question, I implemented the same program using pure C with libxml, which has a SAX API. I won't paste it wholly here because it's longer, but you can find the full source code on GitHub. It takes just 0.56 seconds to process our 230 MiB input file, which is very impressive given the other results, but also not very surprising. This is C, after all.

You may wonder - if lxml uses libxml underneath, why is it so much slower than the pure C version? The answer is Python call overhead. The lxml version calls back into Python for every parsed element, which incurs a significant cost [3]. Another reason is that my C implementation doesn't actually parse an element - it's just a simple event-based state machine, so there's less extra work being done.

Using libxml from Go

To recap where we are so far:

Python libraries based on underlying C implementations are faster than pure Go.
Pure C is much faster still.

We have two options: we can either try to optimize Go's encoding/xml package, or we can try to wrap a fast C library with Go. While the former is a worthy goal, it involves a large effort and should be a topic for a separate post. Here, I'll go for the latter.

Seaching around the web, I found a few wrappers around libxml. Two that seemed moderately popular and maintained are https://github.com/lestrrat-go/libxml2 and https://github.com/moovweb/gokogiri. Unfortunately, neither of these (or the other bindings I found) are exposing the SAX API of libxml; instead, they focus on the DOM API, where the whole document is parsed by the underlying library and a tree is returned. As mentioned above, we need the SAX interface to process huge files.

gosax

It's time to roll our own :-) I wrote the gosax module, which uses Cgo to call into libxml and exposes a SAX interface [4]. Implementing it was an interesting exercise in Cgo, because it requires some non-trivial concepts like registering Go callbacks with C.

Here's a version of our program using gosax:

package main

import (
  "fmt"
  "os"
  "strings"

  "github.com/eliben/gosax"
)

func main() {
  counter := 0
  inLocation := false

  scb := gosax.SaxCallbacks{
    StartElement: func(name string, attrs []string) {
      if name == "location" {
        inLocation = true
      } else {
        inLocation = false
      }
    },

    EndElement: func(name string) {
      inLocation = false
    },

    Characters: func(contents string) {
      if inLocation && strings.Contains(contents, "Africa") {
        counter++
      }
    },
  }

  err := gosax.ParseFile(os.Args[1], scb)
  if err != nil {
    panic(err)
  }

  fmt.Println("counter =", counter)
}

As you can see, it implements a state machine that remembers being inside a location element, where the character data is checked. This program takes 4.03 seconds to process our input file. Not bad! But we can do a bit better, and with a couple of optimizations I managed to bring it down to 3.68 seconds - about the same speed as the Python implementations!

IMHO the roughly similar run times here are a coincidence, because the Python programs are different from my approach in that they expose a higher-level API than pure SAX. Recall that iterparse returns a parsed element, and we can access its text attribute, etc. In gosax, we have to do this much more manually. Since the the cost of calls between Cgo and Go is rather high, there is an optimization opportunity here for gosax. We could do more work in C - parsing a full element, and returning it wholly to Go. This would move work from the Go side to the C side, as well as reduce the number of cross-language calls. But this is a task for another day.

Conclusion

Well, this was fun :-) There are 5 different implementations of the same simple task described here, in 3 different programming languages. Here is a summary of the speed measurements we got:

Python's performance story has always been - "it's probably fast enough, and in the rare cases where it isn't, use a C extension". In Go the narrative is somewhat different: in most cases, the Go compiler produces fairly fast code. Pure Go code is significantly faster than Python and often faster than Java. Even so, every once in a while it may be useful to dip into C or C++ for performance, and in these cases Cgo is a good approach.

It's obvious that encoding/xml needs some work w.r.t. performance, but until that happens - there are good alternatives! Leveraging the speed of libxml has been possible for the DOM API, and now is possible for the SAX API as well. In the long run, I believe that serious performance work on encoding/xml can make it go faster than the libxml wrappers because it would elimitate the high cost of C-to-Go calls.

[1]	This size will easily fit in RAM, but it's good enough to provide a meaningful benchmarking duration.

[2]

When working on optimizations, it's often useful to know "the speed of light" of some computation. Say we want to optimize some function in our program. It's worth asking - how much faster will the program be if this function takes 0 time? If the overall change is tiny, the function is not worth optimizing, most likely. This is just a practical application of Amdahl's law.

[3]

We can test this hypothesis by timing how long it takes the non-streaming API in lxml to parse the same file. Since it parses the whole XML file in C before returning the parsed structure to Python, we expect the Python call overhead to be much smaller. Indeed, for files that fit into memory this is faster. But once again, in this post we return our attention to streaming APIs - assuming this is our only choice for gigantic files.

[4]

gosax is very minimal, only providing the most common SAX callbacks. The decision to create a new module was just for convenience and speed; the more correct thing would have likely been to contribute to one of the existing libxml wrappers. I don't see gosax as production-quality at this stage - I just hacked it together to be able to experiment for this post.