Eli Bendersky's website - Go

Thoughts on starting new projects with LLM agents

2026-06-06T17:38:00-07:00

A few months ago I wrote about using LLM agents to help restructuring one of my Python projects. It's worth beginning by saying that the rewrite has been successful by all reasonable measures; I've been able to continue maintaining that project since then without an issue.

In this post, I want to discuss another project I've recently completed with significant help from agents: watgo. In this project many things are different; most notably, it's a from-scratch project rather than a rewrite, and it uses a different programming language (Go). This post describes my experience working on the project, and some lessons learned along the way.

The process

This is a new project, so it required extensive design. I began by iterating on the design with the agent, with a sketch of the API. For this purpose, I recommend using a Markdown file committed into the repository for future reference.

After that, I started asking the agent to write CLs [1] in a logical order that made sense to me, keeping them small and reviewable (more on this in the next section). Sometimes it's not easy to have a small CL, and multiple rounds of revision may confuse the agent; in this case, I commit the CL and then go back and ask the agent to modify or refactor the code, as much as needed, with separate CLs. In the worst case, the whole sequence can be reverted if I feel we've taken the wrong direction (branches could also be helpful here for more complicated scenarios).

This point is worth reiterating: sometimes a single CL is a huge step forward, but requires lots of review, cleanup and refactoring to be viable. I've had multiple instances where an agent produced several days of work in a single CL, but I then spent hours instructing it to clean up and refactor. Overall, it's still a productivity gain, just not as much as some pundits would like us to believe.

Keeping the human in the loop

Given the current state of agent capabilities, I think it's worth splitting projects into two categories:

Low importance / prototype / throw away projects where deep code understanding is unnecessary. These can be "vibe-coded" (submitting agent code without even reviewing it).
High importance projects that I actually want to maintain; here, vibe-coding is ill advised and I insist on reviewing and guiding all code the agent writes before it's submitted (or shortly after, as discussed above).

The watgo projects is a clear example of (2): I certainly intend to maintain this project in the long term, so I insist on code that I understand. With very few exceptions, no code gets in without full review and often multiple rounds of revisions.

Even if the cost for writing code went down, maintaining a project is so much more than that. It's triaging and fixing bugs, it's thinking through what needs to be done rather than how to do it, it's keeping the code healthy over time, and so on. As Brian Kernighan said:

Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?

Maybe at some point agents will become good enough that projects in category (2) can be implemented and maintained completely autonomously. Maybe. But we're certainly not there yet. My hunch is that getting there will require crossing the AGI line [2], after which little in our world remains certain.

Practical workflow

If you're using an agent to send an actual PR and only review that, it's difficult to be disciplined enough to actually perform a thorough review. I find the following method to be more reliable:

I use a CLI agent running locally in my repository, and ask it to update the code there. In parallel, I have a VSCode window open in the same project, where I can:

Review the agent's changes using VSCode's diff view
Make my own tweaks and code changes if needed

Once I'm pleased with the change, I manually create a commit.

Keeping the CLs small

As mentioned above, it's imperative to keep making progress in small chunks, with small enough CLs that a human can fully understand in a single review. It's very tempting to sprint ahead submitting thousands of lines of code every day, but this temptation has to be avoided. Coding with an agent is like speed-reading; yes, you're making more progress, but comprehension suffers the faster you go.

Particularly for refactoring, agents still take the shortest route to destination. It's important to guide them to think about the "big picture" at all times, find all instances where X is better done as Y, not just a single place noticed during a review. This is why it's sometimes OK to have a CL submitted before you fully agree with everything, and go back to it later for several refactoring rounds. Source control works amazingly well when pair-coding with agents.

Testing strategy

It's a key point discussed in every "how to succeed with AI" article, but still critical enough to reiterate here: a solid testing strategy is absolutely crucial for success. Agents produce - by far - the best results when they have a solid test suite to test their code against.

With the pycparser rewrite, I had a large existing test suite. For watgo, the very first thing I did was think through how to adapt the test suites of the WASM spec and of the wabt project for my needs.

If your project doesn't have such tests to rely on, this should be your first order of business - finding one, or building one from scratch. Beware of self-reinforcing loops though; it's dangerous to trust agents for both the tests and the implementations tested against them.

Language choice - Go for agent-written projects

Go is a fantastic language for agents to write, because it's designed to be very readable by humans. The biggest strengths of Go are exactly what makes the experience of reviewing agent code so positive:

Go changes very infrequently, so you don't have to wonder "are we using the most modern / idiomatic approach" or "what the hell is this construct" as often as with other languages (looking at you, Python and TypeScript).
There are relatively few ways to accomplish the same thing in Go, further lowering the mental burden.
The standard library is rich and there's much less need to keep abreast of the package-everyone-uses du jour.
In general, Go is designed for readability, with a mild-but-still-strong type system, uniform formatting, explicit error propagation and opinionated choices already made for you.

Since most of the time spent by humans when using agents is reading rather than writing code, these effects compound and produce a great experience. Recall the discussion of how some languages are optimized for writability (Perl) while others are optimized for readability (Go)? Well, when working on a project with an agent we live in a world of 99% reading vs. 1% writing, so this really matters.

I find this aspect really crucial in light of the earlier points made in this post - namely, keeping the human in the loop by understanding and reviewing all of the agent's design choices and code.

Final thoughts

If you're working on a subject that's completely new to you, I would strongly recommend against the approach described in this post. To really learn something, you have to work through it from scratch, yourself, reading, designing, writing the code. Agents don't change this basic fact; even before agents, if you wanted to learn X, copying it from Stack Overflow or some other project clearly wasn't the right way to go. Similarly, while agents can be used as a prop for learning, they cannot learn for you.

As a corollary, junior engineers should exercise extreme caution when relying on LLMs. There's no replacement to hard-won experience and the sweat and tears of learning new, challenging topics. Learning is supposed to be hard; if it's too easy, you're probably not learning.

For senior engineers, agents are a boon; it's a great tool to increase productivity, avoid the boring stuff, and get unstuck from procrastination; but only when used judiciously.

[1]	CL stands for Changelist, also known as a "patch" or a "diff" - basically a standalone commit that touches one or more files. This term originates from the source control systems Perforce and Subversion.

[2]	Programming is the ultimate realization of thought; if machines can design, produce, maintain and understand code better than humans, it means they can start improving themselves, which is the definition of singularity.

watgo - a WebAssembly Toolkit for Go

2026-04-09T19:28:00-07:00

I'm happy to announce the general availability of watgo - the WebAssembly Toolkit for Go. This project is similar to wabt (C++) or wasm-tools (Rust), but in pure, zero-dependency Go.

watgo comes with a CLI and a Go API to parse WAT (WebAssembly Text), validate it, and encode it into WASM binaries; it also supports decoding WASM from its binary format.

At the center of it all is wasmir - a semantic representation of a WebAssembly module that users can examine (and manipulate). This diagram shows the functionalities provided by watgo:

Parse: a parser from WAT to wasmir
Validate: uses the official WebAssembly validation semantics to check that the module is well formed and safe
Encode: emits wasmir into WASM binary representation
Decode: read WASM binary representation into wasmir

CLI use case

watgo comes with a CLI, which you can install by issuing this command:

go install github.com/eliben/watgo/cmd/watgo@latest

The CLI aims to be compatible with wasm-tools [1], and I've already switched my wasm-wat-samples projects to use it; e.g. a command to parse a WAT file, validate it and encode it into binary format:

watgo parse stack.wat -o stack.wasm

API use case

wasmir semantically represents a WASM module with an API that's easy to work with. Here's an example of using watgo to parse a simple WAT program and do some analysis:

package main

import (
  "fmt"

  "github.com/eliben/watgo"
  "github.com/eliben/watgo/wasmir"
)

const wasmText = `
(module
  (func (export "add") (param i32 i32) (result i32)
    local.get 0
    local.get 1
    i32.add
  )
  (func (param f32 i32) (result i32)
    local.get 1
    i32.const 1
    i32.add
    drop
    i32.const 0
  )
)`

func main() {
  m, err := watgo.ParseWAT([]byte(wasmText))
  if err != nil {
    panic(err)
  }

  i32Params := 0
  localGets := 0
  i32Adds := 0

  // Module-defined functions carry a type index into m.Types. The function
  // body itself is a flat sequence of wasmir.Instruction values.
  for _, fn := range m.Funcs {
    sig := m.Types[fn.TypeIdx]
    for _, param := range sig.Params {
      if param.Kind == wasmir.ValueKindI32 {
        i32Params++
      }
    }

    for _, instr := range fn.Body {
      switch instr.Kind {
      case wasmir.InstrLocalGet:
        localGets++
      case wasmir.InstrI32Add:
        i32Adds++
      }
    }
  }

  fmt.Printf("module-defined funcs: %d\n", len(m.Funcs))
  fmt.Printf("i32 params: %d\n", i32Params)
  fmt.Printf("local.get instructions: %d\n", localGets)
  fmt.Printf("i32.add instructions: %d\n", i32Adds)
}

One important note: the WAT format supports several syntactic niceties that are flattened / canonicalized when lowered to wasmir. For example, all folded instructions are lowered to unfolded ones (linear form), function & type names are resolved to numeric indices, etc. This matches the validation and execution semantics of WASM and its binary representation.

These syntactic details are present in watgo in the textformat package (which parses WAT into an AST) and are removed when this is lowered to wasmir. The textformat package is kept internal at this time, but in the future I may consider exposing it publicly - if there's interest.

Testing strategy

Even though it's still early days for watgo, I'm reasonably confident in its correctness due to a strategy of very heavy testing right from the start.

WebAssembly comes with a large official test suite, which is perfect for end-to-end testing of new implementations. The core test suite includes almost 200K lines of WAT files that carry several modules with expected execution semantics and a variety of error scenarios exercised. These live in specially designed .wast files and leverage a custom spec interpreter.

watgo hijacks this approach by using the official test suite for its own testing. A custom harness parses .wast files and uses watgo to convert the WAT in them to binary WASM, which is then executed by Node.js [2]; this harness is a significant effort in itself, but it's very much worth it - the result is excellent testing coverage. watgo passes the entire WASM spec core test suite.

Similarly, we leverage wabt's interp test suite which also includes end-to-end tests, using a simpler Node-based harness to test them against watgo.

Finally, I maintain a collection of realistic program samples written in WAT in the wasm-wat-samples repository; these are also used by watgo to test itself.

[1]	Though not all of wasm-tools's functionality is supported yet.

[2]	To stick to a pure-Go approach also for testing, I originally tried using wazero for this, but had to give up because wazero doesn't support some of the recent WASM proposals that have already made it into the standard (most notably Garbage Collection).

Consistent hashing

2025-09-27T05:54:00-07:00

This post is an introduction to consistent hashing, an algorithm for designing a hash table such that only a small portion of keys has to be recomputed when the table's size changes.

Motivating use case

Suppose we're designing a caching web proxy, but the expected storage demands are higher than what a single machine can handle. So we distribute the cache across multiple machines. How do we do that? Given a URL, how do we make sure that we can easily find out which server we should approach for a potentially cached version [1]?

An approach that immediately comes to mind is hashing. Let's calculate a numeric hash of the URL and distribute it evenly between N nodes (that's what we'll call the servers in this post):

hash := calculateHashFunction(url)
nodeId := hash % N

This process works but turns out to have serious downsides in real-world applications.

The problem with the naive hashing approach

Consider our caching use case again; in a realistic application at "internet scale", one of the assumptions we made implicitly doesn't hold - the cache nodes are not static. New nodes are added to the system if the load is high (or if new machines come into service); existing nodes can crash or be taken offline for maintenance. In other words, the number N in our application is not a constant.

The problem may be apparent now; to demonstrate it directly, let's take an actual implementation of hashItem using Go's md5 package [2]:

// hashItem computes the slot an item hashes to, given a total number of slots.
func hashItem(item string, nslots uint64) uint64 {
  digest := md5.Sum([]byte(item))
  digestHigh := binary.BigEndian.Uint64(digest[8:16])
  digestLow := binary.BigEndian.Uint64(digest[:8])
  return (digestHigh ^ digestLow) % nslots
}

The terminology is slightly adjusted:

Instead of url, we'll just refer to a generic item
"Slot" is a common concept in hash tables: our hashItem computes a slot number for an item, given the total number of available slots

Let's say we started with 32 slots, and we hashed the strings "hello", "consistent" and "marmot". We get these slots:

hello       (n=32): 4
consistent  (n=32): 14
marmot      (n=32): 5

Now suppose that another node is added, and the total nslots grows to 33. Hashing our items again:

hello       (n=33): 23
consistent  (n=33): 18
marmot      (n=33): 31

All the slots changed!

This is a significant problem with the naive hashing approach. Whenever nslots changes, we get completely different slots for pretty much any item. In a realistic application it means that whenever a new node joins or leaves our caching cluster, there will be a flood of cache misses on every query until the new cluster settles. And node changes sometimes occur at the most incovenient times; imagine that the load is spiking (maybe a site was mentioned in a high-profile news outlet, or there's a live event streaming) and new nodes are added to handle it. This isn't a great time to temporarily lose all caching!

Consistent hashing

The consistent hashing algorithm solves the problem in an elegant way. The key idea is to map both nodes and items onto an interval, and then an item belongs to a node closest to it. Concretely, we take the unit circle [3], and map nodes and items to angles on this circle. Here's an example that explains how this method works in more detail:

This shows five nodes: N1 through N5, and three items: Ix, Iy, Iz. Initially, we add the nodes: using a hashing operation we map them onto the circle (details later). Then, as items come in, we determine which node they belong to, as follows:

Use the same hashing operation to find the item's location on the circle
The node this item belongs to is the closest one, in the clockwise direction

In our diagram, Ix is mapped to N1, Iy to N2, and Iz to N3. So far so good, but the benefit of this approach becomes apparent when the nodes change. In our diagram, suppose N3 is removed. Then Iz will map to N5. The mapping of the other items doesn't change!

Adding nodes has a similar outcome. If a new node N6 is added and it hashes to a position between Iy and N2 on the circle, from that moment Iy will be mapped to N6, but the other items keep their mapping.

Suppose we have a total of M items that we need to distribute across N nodes. Using the naive hashing approach, whenever we add or remove a node, all M items change their mapping. On the other hand, with consistent hashing only about \frac{M}{N} need to change. This is a huge difference.

The original consistent hashing paper (see [1]) calls this the monotonicity property of the algorithm:

If items are initially assigned to a set of buckets V_1, and then some new buckets are added to form V_2, then an item may move from an old bucket to a new bucket, but not from one old bucket to another.

Implementing consistent hashing

Implementing the consistent hashing algorithm as described above is fairly easy. The most critical part of the implementation is finding which node an item maps to - this involves some kind of search. The original consistent hashing paper suggests using a balanced binary tree for the search; the implementation I'm demonstrating here uses a slightly different but equivalent approach: binary search in a linear array of node positions (slots) [4].

First, some practical considerations:

Theoretically, the unit circle can be seen as the continuous range [0, 1). In programming we much prefer the discrete domain, however, so we're going to "quantize" this range to [0, ringSize), where ringSize is some suitably large number that avoids collisions.
Looking at the circle diagram above, imagine that 0 degrees is the "north" direction (12 o'clock), and angles increase clockwise. In our discrete domain, 12 o'clock is 0, 3 o'clock is ringSize/4, and so on.

When a node is added to the consistent hash, its location is found by applying a hash function like hashItem as described above, with nslots=ringSize. The nodes are stored using a pair of data structures, as follows; this example uses the approximate locations of the nodes N1 through N5 in the circle diagram above (assume ringSize=1024 here):

The positions of the nodes on the circle are stored in slots, which is sorted. nodes holds the corresponding node names. For each i, nodes[i] is at position slots[i] on the circle.

Here's the ConsistentHasher data structure in Go:

type ConsistentHasher struct {
  // nodes is a list of nodes in the hash ring; it's sorted in the same order
  // as slots: for each i, the node at index slots[i] is nodes[i].
  nodes []string

  // slots is a sorted slice of node indices.
  slots []uint64

  ringSize uint64
}

// NewConsistentHasher creates a new consistent hasher with a given maximal
// ring size.
func NewConsistentHasher(ringSize uint64) *ConsistentHasher {
  return &ConsistentHasher{
    ringSize: ringSize,
  }
}

And this is how finding which node a given item maps to is implemented:

// FindNodeFor finds the node an item hashes to. It's an error to call this
// method if the hasher doesn't have any nodes.
func (ch *ConsistentHasher) FindNodeFor(item string) string {
  if len(ch.nodes) == 0 {
    panic("FindNodeFor called when ConsistentHasher has no nodes")
  }
  ih := hashItem(item, ch.ringSize)

  // Since ch.slots is a sorted list of all the node indices for our nodes, a
  // binary search is what we need here. ih is mapped to the node that has the
  // same or the next larger node index. slices.BinarySearch does exactly this,
  // by returning the index where the value would be inserted.
  slotIndex, _ := slices.BinarySearch(ch.slots, ih)

  // When the returned index is len(slots), it means the search wrapped
  // around.
  if slotIndex == len(ch.slots) {
    slotIndex = 0
  }

  return ch.nodes[slotIndex]
}

The key here is the binary search invocation. Adding and removing nodes is done similarly using binary search - see the full code.

Better item distribution with virtual nodes

A common issue that comes up in the implementation of consistent hashing is unbalanced distribution of items across the different nodes. With M items and a total of N nodes, the average distribution will be about \frac{M}{N} per node, but in practice it won't be very balanced - some nodes will have many more items assigned to them than others (see the Appendix for more details).

In a real application, this may mean that some cache servers will be much busier than others, which is a bad thing as far as capacity planning and efficient use of HW. Luckily, there's an elegant tweak to the consistent hashing algorithm that significantly mitigates the problem: virtual nodes.

Instead of mapping each node to a single location on the circle, we'll map it to V locations instead. There are several ways to do this - the simplest is just to tweak the node name in some way. For example, when AddNode is called to add node, it will run:

for i := range V {
  vnodeName = fmt.Sprintf("%v@%v", node, i)

  // ... now add vnodeName to the nodes/slots slices
}

Then, when looking up an item we'll run into one of the virtual nodes, decode the node's name from it (in our example just strip the @<number> suffix) and return that. Implementing node removal is similarly simple.

The idea is that given a node named foo, the virtual node names foo@0, foo@1, foo@2 etc. will be spread all around the circle and not cluster in a single place. See the Appendix for a calculation of how this affects the final distribution.

The source code for this post includes a ConsistentHasherV type that is very similar to ConsistentHasher, except that it implements the virtual node strategy. The user interface remains exactly the same - it's only the internal implementation that changes slightly.

Code

The full source code for this post is on GitHub.

Appendix

The quality of the hash function is very important for good shuffling of nodes on the circle, but even if we take a perfect hash function that produces uniformly distributed values, the outcome is likely to be sub-optimal for our needs.

Let's say we select N points on the unit circle, uniformly in the range [0, 1). If we sort the points by angle, the gaps between neighboring angles are the order statistics. These follow the Beta distribution with parameters (1, N-1), which has a mean of \frac{1}{N} and a variance of \frac{N-1}{N^2(N+1)}.

This is quite significant. Consider a circle with 20 nodes. The standard deviation of the distribution is the square root of the variance; substituting N=20, we get:

\[\sigma=\sqrt{\frac{19}{20^2\cdot 21}}=0.048\]

With 20 nodes uniformly distributed around a circle, we can expect an average of 18 degrees distance between two nodes. A standard deviation of 0.048 means 17 degrees, which is comparable to the average!

We can also do a realistic example to demonstrate this. Let's generate 20 random angles on a circle, and show how the node distribution looks:

In this particular sample, the average angle between two adjacent nodes is 18 degrees (as expected). The smallest angle is just 1.04 degrees, while the largest one is 42 degrees. This means that some nodes will get 40x as many items assigned to them as others!

It's easy to see how virtual nodes help; imagine that each server maps to some number of randomly distributed nodes on the circle; some of these will be farther than others from their closest neighbor, but the average will have much less variety. Mathematically, given a set of n uniformly distributed random variables with variance v, the variance of their average is \frac{v}{n}.

As a concrete experiment, I ran a similar simulation to the one above, but with 10 virtual nodes per node. We'll consider the total portion of the circle mapped to a node when it maps to either of its virtual nodes. While the average remains 18 degrees, the variance is reduced drastically - the smallest one is 11 degrees and the largest 26.

You can find the code for these experiments in the demo.go file of the source code repository.

[1]

This is close to the original motivation for the development of consistent caching by researchers at MIT. Their work, described in the paper "Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web" became the foundation of Akamai Technologies.

There are other use cases of consistent hashing in distributed systems. AWS popularized one common application in their paper "Dynamo: Amazon’s Highly Available Key-value Store", where the algorithm is used to distribute storage keys across servers.

[2] In a real application, we'd probably want to find a different hash function. md5 is relatively slow, and we don't need any of its cryptographic guarantees in this use case. That said, the chosen hash function should be suitable and produce a good mixing of bits even for items that are only slightly different (e.g. "node-1" vs. "node-2"); I found that Go's built-in hash/fnv package isn't great for this purpose, for example.

[3]	A circle is used instead of a linear range to conveniently handle boundary conditions. It's similar to using modular arithmetic for the naive hashing approach.

[4]

I didn't bother to compare performance, but I suspect the array-based approach will be faster for lookup because of being relatively cache-friendly. For insertion/removal, the array approach is O(n) whereas the tree is O(\log\ n), but it's safe to assume that lookup performance is significantly more important. Nodes don't change very often, and lookups happen orders of magnitude more frequently.

Implementing Forth in Go and C

2025-08-26T20:38:00-07:00

I first ran into Forth about 20 years ago when reading a book about designing embedded hardware. The reason I got the book back then was to actually learn more about the HW aspects, so having skimmed the Forth chapter I just registered an "oh, this is neat" mental note and moved on with my life. Over the last two decades I heard about Forth a few more times here and there, such as that time when Factor was talked about for a brief period, maybe 10-12 years ago or so.

It always occupied a slot in the "weird language" category inside my brain, and I never paid it much attention. Until June this year, when a couple of factors combined fortuitously:

After spending much of the earlier part of 2025 exploring the inner workings of LLMs and digging in random mathy and algorithmic topics, I had an itch to just write some code.
I somehow found Dave Gauer's page about Forth and also the one on Implementing a Forth.

And something clicked. I'm going to implement a Forth, because... why not?

So I spent much of my free hacking time over the past two months learning about Forth and implementing two of them.

Forth: the user level and the hacker level

It's useful to think of Forth (at least standard Forth, not offshoots like Factor) as having two different "levels":

User level: you just want to use the language to write programs. Maybe you're indeed bringing up new hardware, and find Forth a useful calculator + REPL + script language. You don't care about Forth's implementation or its soul, you just want to complete your task.
Hacker level: you're interested in the deeper soul of Forth. Isn't it amazing that even control flow constructs like IF...THEN or loops like BEGIN...UNTIL are just Forth words, and if you wanted, you could implement your own control flow constructs and have them be first-class citizens, as seamless and efficient as the standard ones?

Another way to look at it (useful if you belong to a certain crowd) is that user-level Forth is like Lisp without macros, and hacker-level Forth has macros enabled. Lisp can still be great and useful without macros, but macros take it to an entire new level and also unlock the deeper soul of the language.

This distinction will be important when discussing my Forth implementations below.

goforth and ctil

There's a certain way Forth is supposed to be implemented; this is how it was originally designed, and if you get closer to the hacker level, it becomes apparent that you're pretty much required to implement it this way - otherwise supporting all of the language's standard words will be very difficult. I'm talking about the classical approach of a linked dictionary, where a word is represented as a "threaded" list [1], and this dictionary is available for user code to augment and modify. Thus, much of the Forth implementation can be written in Forth itself.

The first implementation I tried is stubbornly different. Can we just make a pure interpreter? This is what goforth is trying to explore (the Go implementation located in the root directory of that repository). Many built-in words are supported - definitely enough to write useful programs - and compilation (the definition of new Forth words using : word ... ;) is implemented by storing the actual string following the word name in the dictionary, so it can be interpreted when the word is invoked.

This was an interesting approach and in some sense, it "works". For the user level of Forth, this is perfectly usable (albeit slow). However, it's insufficient for the hacker level, because the host language interpreter (the one in Go) has all the control, so it's impossible to implement IF...THEN in Forth, for example (it has to be implemented in the host language).

That was a fun way to get a deeper sense of what Forth is about, but I did want to implement the hacker level as well, so the second implementation - ctil - does just that. It's inspired by the jonesforth assembly implementation, but done in C instead [2].

ctil actually lets us implement major parts of Forth in Forth itself. For example, variable:

: variable create 1 cells allot ;

Conditionals:

\ IF, ELSE, THEN work together to compile to lower-level branches.
\
\ IF ... THEN compiles to:
\   0BRANCH OFFSET true-part rest
\ where OFFSET is the offset of rest
\
\ IF ... ELSE ... THEN compiles to :
\   0BRANCH OFFSET true-part BRANCH OFFSET2 false-part rest
\ where OFFSET is the offset of false-part and OFFSET2 is the offset of rest
: if immediate
  ' 0branch ,
  here
  0 ,
  ;

: then immediate
  dup
  here swap -
  swap
  ! ;

: else immediate
  ' branch ,
  here
  0 ,
  swap
  dup
  here swap -
  swap
  ! ;

These are actual examples of ctil's "prelude" - a Forth file loaded before any user code. If you understand Forth, this code is actually rather mind-blowing. We compile IF and the other words by directly laying our their low-level representation in memory, and different words communicate with each other using the data stack during compilation.

Thoughts on Forth itself

Forth made perfect sense in the historic context in which it was created in the early 1970s. Imagine having some HW connected to your computer (a telescope in the case of Forth's creator), and you have to interact with it. In terms of languages at your disposal - you don't have much, even BASIC wasn't invented yet. Perhaps your machine still didn't have a C compiler ported to it; C compilers aren't simple, and C isn't very great for exploratory scripting anyway. So you mostly just have your assembly language and whatever you build on top.

Forth is easy to implement in assembly and it gives you a much higher-level language; you can use it as a calculator, as a REPL, and as a DSL for pretty much anything due to its composable nature.

Forth certainly has interesting aspects; it's a concatenative language, and thus inherently point-free. A classical example is that instead of writing the following in a more traditional syntax:

eat(bake(prove(mix(ingredients))))

You just write this:

ingredients mix prove bake eat

There is no need to explicitly pass parameters, or to explicitly return results. Everything happens implicitly on the stack.

This is useful for REPL-style programming where you use your language not necessarily for writing large programs, but more for interactive instructions to various HW devices. This dearth of syntax is also what makes Forth simple to implement.

All that said, in my mind Forth is firmly in the "weird language" category; it's instructive to learn and to implement, but I wouldn't actually use it for anything real these days. The stack-based programming model is cool for very terse point-free programs, but it's not particularly readable and hard to reason about without extensive comments, in my experience.

Consider the implementation of a pretty standard Forth word: +!. It expects and address at the top of stack, and an addend below it. It adds the addend to the value stored at that address. Here's a Forth implementation from ctil's prelude:

: +!        ( addend addr -- )
  tuck      ( addr addend addr )
  @         ( addr addend value-at-addr )
  +         ( addr updated-value )
  swap      ( updated-value addr )
  ! ;

Look at that stack wrangling! It's really hard to follow what goes where without the detailed comments showing the stack layout on the right of each instruction (a common practice for Forth programs). Sure, we can create additional words that would make this simpler, but that just increases the lexicon of words to know.

My point is, there's fundamental difficulty here. When you see this C code:

int func(int a, int b) {
  return foo(a, bar(b));
}

Even without any documentation, you can immediately know several important things:

bar has one parameter and one return value
foo has two parameters and one return value
func also has two parameters and one return value
It's immediately obvious how the various values flow from one function call to the next.

Written in Forth [3]:

: func bar foo ;

How can you know the arity of the functions without adding explicit comments? Sure, if you have a handful of words like bar and foo you know like the back of your hand, this is easy. But imagine reading a large, unfamiliar code base full of code like this and trying to comprehend it.

Summary and links

The source code of my goforth project is on GitHub; both implementations are there, with a comprehensive test harness that tests both.

The learn Forth itself, I found these resources very useful:

Dave Gauer's Forth page
Starting Forth - a free online book / tutorial on Forth for beginners

To learn how to implement Forth:

Dave Gauer's page on Implementing a Forth
jonesforth implementation
Threaded Interpretive Languages - an old but nice book that explains how Forth implementations typically work

Implementing Forth is a great self-improvement project for a coder; there's a pleasantly challenging hump of understanding to overcome, and you gain valuable insights into stack machines, interpretation vs. compilation and mixing these levels of abstraction in cool ways.

Also, implementing programming languages from scratch is fun! It's hard to beat the feeling of getting to interact with your implementation for the first time, and then iterating on improving it and making it more featureful. Just one more word!

[1]	This has nothing to do with threads in the sense of concurrency. Rather, it's thread like in sewing, where the elements of the list are all connected to each other as if with a thread. See this page for more details.

[2]

Which is another deviation from the norm. Forth is really supposed to be implemented in assembly - this is what it was designed for, and it's very clear from its structure that it must be so in order to achieve peak performance.

But where's the fun in doing things the way they were supposed to be done? Besides, jonesforth is already a perfectly fine Forth implementation in assembly, so I wouldn't have learned much by just copying it.

I had a lot of fun coding in C for this one; it's been a while since I last wrote non-trivial amounts of C, and I found it very enjoyable.

[3]	Assuming the convention that multi-parameter functions have their parameters pushed to the stack from left to right.

Bloom filters

2025-05-01T18:35:00-07:00

The original motivation for the creation of Bloom filters is efficient set membership, using a probabilistic approach to significantly reduce the time and space required to reject items that are not members in a certain set.

The data structure was proposed by Burton Bloom in a 1970 paper titled "Space/Time Trade-offs in Hash Coding with Allowable Errors". It's a good paper that's worth reading.

Why Bloom filters?

Suppose that we store some information on disk and want to check if a certain file contains a certain entry. Reading from disk is time consuming, so we want to minimize it as much as possible. A Bloom filter is a data structure that implements a cache with probabilistic properties:

If the cache says the key is not present in a specific file, then it's 100% certain we should not be reading the file.
If the cache says the key is present in the file, there's a small chance this is a false positive (and in fact the key isn't there). In this case we just read the file as usual.

In a scenario where the majority of queries "is this key in that file?" have a negative answer, a Bloom filter can significantly speed up the system [1]. Moreover, the probabilistic nature (the existence of false positives) allows Bloom filters to be extremely fast and occupy very little space. Here's a quote from the Bloom paper:

The new hash-coding methods to be introduced are suggested for applications in which the great majority of messages to be tested will not belong to the given set. For these applications, it is appropriate to consider as a unit of time (called reject time) the average time required to classify a test message as a nonmember of the given set.

How a Bloom filter works

A Bloom filter is a special kind of a hash table with open addressing. It's an array of bits (the length is typically denoted m), and some fixed number (k) of hash functions. We'll assume each hash function can take an arbitrary sequence of bytes and hash it into an integer in the inclusive range [0, m-1]. A Bloom filter supports two operations:

Insert an item: the item is hashed using each of the k hash functions, and the appropriate bits in the underlying array are set to 1.

Test if an item is a member: the item is hashed using each of the k hash functions. If any of the bits indicated by their results is 0, we return "false" with certainty. If all the bits are 1, we return "true" - and there's a small chance of false positives.

Here's how the Bloom paper describes it:

The hash area is considered as N individual addressable bits, with addresses 0 through N - 1. It is assumed that all bits in the hash area are first set to 0.

Next, each message in the set to be stored is hash coded into a number of distinct bit addresses, say a1, a2, . . . , ad. Finally, all d bits addressed by a1 through ad are set to 1.

To test a new message a sequence of d bit addresses, say a'1, a'2, ... a'd, is generated in the same manner as for storing a message. If all d bits are 1, the new message is accepted. If any of these bits is zero, the message is rejected.

Hopefully it's clear why this data structure is probabilistic in nature: it's possible that different items hash to the same number, and therefore when we test some X, all its hashes point to bits turned on by the hashing of other data. Read the Math appendix for the math behind Bloom filters and how to calculate (and design for a specific) the false positive rate.

Here's an example:

We start with an empty bloom filter with m=16 and k=3. All bits are initialized to 0.
Insertion of "x". The three hashes return indices 1, 6, 15, so these bits in the array are set to 1.
Insertion of "y". Hashing returns indices 6, 9 and 13, so these bits in the array are set to 1. Note that bit 6 is set for both "x" and "y", and that's fine.

Next, let's look at some membership tests:

Test "x". Hashing returns 1, 6, 15; all these bits are 1 in the array, so the answer is "true". This is a true positive.
Test "w". Hashing returns 3, 9, 13. Since the bit at position 3 is 0, the answer is "false".
Test "v". Hashing returns 9, 13, 15; all these bits are 1 in the array, so the answer is "true". This is a false positive.

Note that it's trivial to prove (by the law of contraposition) that all "false" answers from a Bloom filter's test operation are true negatives.

Implementation

Here's a simple implementation of a Bloom filter in Go:

// New creates a new BloomFilter with capacity m, using k hash functions.
// You can calculate m and k from the number of elements you expect the
// filter to hold and the desired error rate using CalculateParams.
func New(m uint64, k uint64) *BloomFilter {
  return &BloomFilter{
    m:      m,
    k:      k,
    bitset: newBitset(m),
    seed1:  maphash.MakeSeed(),
    seed2:  maphash.MakeSeed(),
  }
}

type BloomFilter struct {
  m      uint64
  k      uint64
  bitset []uint64

  // seeds for the double hashing scheme.
  seed1, seed2 maphash.Seed
}

// Insert a data item into the bloom filter.
func (bf *BloomFilter) Insert(data []byte) {
  h1 := maphash.Bytes(bf.seed1, data)
  h2 := maphash.Bytes(bf.seed2, data)
  for i := range bf.k {
    loc := (h1 + i*h2) % bf.m
    bitsetSet(bf.bitset, loc)
  }
}

// Test if the given data item is in the bloom filter. If Test returns false,
// it's guaranteed that data was never added to the filter. If it returns true,
// there's an eps probability of this being a false positive. eps depends on
// the parameters the filter was created with (see CalculateParams).
func (bf *BloomFilter) Test(data []byte) bool {
  h1 := maphash.Bytes(bf.seed1, data)
  h2 := maphash.Bytes(bf.seed2, data)
  for i := range bf.k {
    loc := (h1 + i*h2) % bf.m
    if !bitsetTest(bf.bitset, loc) {
      return false
    }
  }
  return true
}

The bitsetSet and bitsetTest functions can be seen in the full code repository.

This implementation uses double hashing to generate k different hash functions from just two hashes.

The code also mentions the CalculateParams function:

// CalculateParams calculates optimal parameters for a Bloom filter that's
// intended to contain n elements with error (false positive) rate eps.
func CalculateParams(n uint64, eps float64) (m uint64, k uint64) {
  // The formulae we derived are:
  // (m/n) = -ln(eps)/(ln(2)*ln(2))
  // k = (m/n)ln(2)

  ln2 := math.Log(2)
  mdivn := -math.Log(eps) / (ln2 * ln2)
  m = uint64(math.Ceil(float64(n) * mdivn))
  k = uint64(math.Ceil(mdivn * ln2))
  return
}

You'll have to read the Math appendix to understand how it works.

Practicalities

Let's look at a practical example of a realistic Bloom filter and how it performs. Suppose we want to store about 1 billion items, and have a false positive rate of 1% (meaning that if the filter returns "true" for a test, there's a 99% chance that the item was previously added to the filter). Using these requirements, we can invoke CalculateParams to get the Bloom filter parameters:

CalculateParams(1000000000, 0.01) ===> (9585058378 7)

This means m is about 9.6 billion (bits) and k is 7. In other words, our Bloom filter requires about 1.2 GB of space to cache the membership test of a billion items (that could be of arbitrary size). Moreover, the lookup is very fast - it's just 7 applications of the hash function. The constant lookup cost is particularly attractive, as it doesn't depend on the number of items actually inserted, or on any particular pattern in the data (i.e. there are no worst case scenarios with asymptotically higher cost).

On my machine, benchmarking with the Go implementation shown above I get ~80 nanoseconds per lookup. Mind you, this is the simplest Go implementation I could think of - nothing here is optimized; I'm sure this can be improved at least 2x by using a more speed-optimized hash implementation, for example.

Now imagine how long it would take to ascertain if data is present in a file with 1 billion entries, even if the file contains proper indexing for fast lookups. Just asking the OS to read the file's first few KiBs to get at the index would take orders of magnitude longer than 80 ns.

Recall that Bloom filters are best suited for cases "in which the great majority of messages to be tested will not belong to the given set". Moreover, even if the data exists in the file, false positives only happen 1% of the time. Therefore, the number of times we'll have to go to the disk just to find the data is not there is a very small fraction of total accesses.

Code

The full code for this post, with tests, is available on GitHub.

Appendix: the Math behind Bloom filters

A reminder on notation:

m: size (in bits) of the set
n: how many keys were inserted into the filter
k: number of hash functions used to insert/test each key

For a specific bit in the set, assuming our hash functions distribute the keys randomly, the probability of it not being set by a specific hash function is:

\[p_0=1-\frac{1}{m}\]

And the probability it’s not set by either of our k hash functions is:

\[p_0=\left ( 1-\frac{1}{m}\right )^k=\left( \left ( 1-\frac{1}{m}\right )^m\right)^\frac{k}{m}\]

The last formula is constructed to use an approximation of e^x for a large enough m (see the appendix in this post) to write:

\[p_0\approx e^{-\frac{k}{m}}\]

After inserting n elements, the probability that it’s 0 is:

\[p_0\approx e^{-\frac{kn}{m}}\]

Meaning that the probability of it being 1 is:

\[p_1\approx 1-e^{-\frac{kn}{m}}\]

Recap: this is the probability of any given bit being 1 after n bits were inserted into a set of size m with k different hash functions.

Assuming independence between our hash functions (this is not super rigorous, but a reasonable assumption in practice), let’s calculate the false positive rate. Suppose we have a new key that’s not in the set, and we’re trying to check its membership by hashing it with our k hash functions. The false positive rate is the probability that all hashes land on a bit that’s already set to 1:

\[p_{fp}\approx\left ( 1-e^{-\frac{kn}{m}} \right )^k\approx \varepsilon\]

This is also called the error rate of our filter, or \varepsilon. To get an optimal (minimal) false positive rate, let’s minimize \varepsilon. Since the logarithm function is monotonically increasing, it will be more convenient to minimize \ln(\varepsilon):

\[ln(\varepsilon)=\ln\left ( 1-e^{-\frac{kn}{m}} \right )^k=k\cdot \ln\left ( 1-e^{-\frac{kn}{m}} \right )\]

We’ll calculate the derivative w.r.t. k and set it to 0:

\[\begin{aligned} \frac{d}{dk}ln(\varepsilon)&=\frac{d}{dk}k\cdot \ln\left ( 1-e^{-\frac{kn}{m}} \right )\\ &=\ln\left ( 1-e^{-\frac{kn}{m}} \right ) + k\frac{e^{-\frac{kn}{m}}\cdot \frac{n}{m}}{1-e^{-\frac{kn}{m}}} \end{aligned}\]

Substituting a variable t=e^{-\frac{kn}{m}} and using some more calculus and algebra, we can find that:

\[k= \frac{m}{n}\cdot \ln(2)\]

A numerical example: if we have a set with 1 million bits, and we expect to insert about 100,000 elements, the optimal number of hash functions is:

\[k= 10\cdot \ln(2)= 6.93 \approx 7\]

However, it’s more useful to aim for a certain error rate, and set the filter parameters accordingly. Let’s assume we’ll be using this optimal value of k. Substituting k= \frac{m}{n}\cdot \ln(2) into the equation for \varepsilon from above:

\[\begin{aligned} \varepsilon&\approx \left ( 1-e^{-\frac{n}{m}\cdot{\frac{m}{n}\cdot \ln 2}} \right )^{\frac{m}{n}\cdot \ln 2}\\ &\approx \left ( 1-e^{-\ln 2} \right )^{\frac{m}{n}\cdot \ln 2}\\ &\approx \left ( \frac{1}{2} \right )^{\frac{m}{n}\cdot \ln 2}\\ \end{aligned}\]

If we use the numerical example from before with \frac{m}{n}=10, the error rate with an optimal k will be 0.5^{6.93}\approx 0.8\%.

What often happens is that we have an error rate in mind and we want to calculate how many bits per element we want to dedicate in our set. Let’s take the previous equation and try to isolate \frac{m}{n} from it using a logarithm:

\[\ln \varepsilon\approx \frac{m}{n}\cdot \ln 2 \cdot \ln 2^{-1}=-\frac{m}{n}\ln^2(2)\]

Then:

\[\frac{m}{n}\approx -\frac{\ln \varepsilon}{\ln^2(2)}\]

Final numerical example: suppose we want an error (false positive) rate of 1%. This means our set should have:

\[\frac{m}{n}\approx -\frac{\ln 0.01}{\ln^2 (2)}=9.58\]

... bits per element. So if we expect about 100,000 elements, the bit set used for our filter should have at least 958,000 bits. And, as calculated earlier, we should be using k=7 hash functions to achieve this optimal error rate.

[1]	For this reason, Bloom filters are very common in data storage systems. Here's a discussion about Cassandra, but there are many others.