Eli Bendersky's website - Programming

Consistent hashing

2025-09-27T05:54:00-07:00

This post is an introduction to consistent hashing, an algorithm for designing a hash table such that only a small portion of keys has to be recomputed when the table's size changes.

Motivating use case

Suppose we're designing a caching web proxy, but the expected storage demands are higher than what a single machine can handle. So we distribute the cache across multiple machines. How do we do that? Given a URL, how do we make sure that we can easily find out which server we should approach for a potentially cached version [1]?

An approach that immediately comes to mind is hashing. Let's calculate a numeric hash of the URL and distribute it evenly between N nodes (that's what we'll call the servers in this post):

hash := calculateHashFunction(url)
nodeId := hash % N

This process works but turns out to have serious downsides in real-world applications.

The problem with the naive hashing approach

Consider our caching use case again; in a realistic application at "internet scale", one of the assumptions we made implicitly doesn't hold - the cache nodes are not static. New nodes are added to the system if the load is high (or if new machines come into service); existing nodes can crash or be taken offline for maintenance. In other words, the number N in our application is not a constant.

The problem may be apparent now; to demonstrate it directly, let's take an actual implementation of hashItem using Go's md5 package [2]:

// hashItem computes the slot an item hashes to, given a total number of slots.
func hashItem(item string, nslots uint64) uint64 {
  digest := md5.Sum([]byte(item))
  digestHigh := binary.BigEndian.Uint64(digest[8:16])
  digestLow := binary.BigEndian.Uint64(digest[:8])
  return (digestHigh ^ digestLow) % nslots
}

The terminology is slightly adjusted:

Instead of url, we'll just refer to a generic item
"Slot" is a common concept in hash tables: our hashItem computes a slot number for an item, given the total number of available slots

Let's say we started with 32 slots, and we hashed the strings "hello", "consistent" and "marmot". We get these slots:

hello       (n=32): 4
consistent  (n=32): 14
marmot      (n=32): 5

Now suppose that another node is added, and the total nslots grows to 33. Hashing our items again:

hello       (n=33): 23
consistent  (n=33): 18
marmot      (n=33): 31

All the slots changed!

This is a significant problem with the naive hashing approach. Whenever nslots changes, we get completely different slots for pretty much any item. In a realistic application it means that whenever a new node joins or leaves our caching cluster, there will be a flood of cache misses on every query until the new cluster settles. And node changes sometimes occur at the most incovenient times; imagine that the load is spiking (maybe a site was mentioned in a high-profile news outlet, or there's a live event streaming) and new nodes are added to handle it. This isn't a great time to temporarily lose all caching!

Consistent hashing

The consistent hashing algorithm solves the problem in an elegant way. The key idea is to map both nodes and items onto an interval, and then an item belongs to a node closest to it. Concretely, we take the unit circle [3], and map nodes and items to angles on this circle. Here's an example that explains how this method works in more detail:

This shows five nodes: N1 through N5, and three items: Ix, Iy, Iz. Initially, we add the nodes: using a hashing operation we map them onto the circle (details later). Then, as items come in, we determine which node they belong to, as follows:

Use the same hashing operation to find the item's location on the circle
The node this item belongs to is the closest one, in the clockwise direction

In our diagram, Ix is mapped to N1, Iy to N2, and Iz to N3. So far so good, but the benefit of this approach becomes apparent when the nodes change. In our diagram, suppose N3 is removed. Then Iz will map to N5. The mapping of the other items doesn't change!

Adding nodes has a similar outcome. If a new node N6 is added and it hashes to a position between Iy and N2 on the circle, from that moment Iy will be mapped to N6, but the other items keep their mapping.

Suppose we have a total of M items that we need to distribute across N nodes. Using the naive hashing approach, whenever we add or remove a node, all M items change their mapping. On the other hand, with consistent hashing only about \frac{M}{N} need to change. This is a huge difference.

The original consistent hashing paper (see [1]) calls this the monotonicity property of the algorithm:

If items are initially assigned to a set of buckets V_1, and then some new buckets are added to form V_2, then an item may move from an old bucket to a new bucket, but not from one old bucket to another.

Implementing consistent hashing

Implementing the consistent hashing algorithm as described above is fairly easy. The most critical part of the implementation is finding which node an item maps to - this involves some kind of search. The original consistent hashing paper suggests using a balanced binary tree for the search; the implementation I'm demonstrating here uses a slightly different but equivalent approach: binary search in a linear array of node positions (slots) [4].

First, some practical considerations:

Theoretically, the unit circle can be seen as the continuous range [0, 1). In programming we much prefer the discrete domain, however, so we're going to "quantize" this range to [0, ringSize), where ringSize is some suitably large number that avoids collisions.
Looking at the circle diagram above, imagine that 0 degrees is the "north" direction (12 o'clock), and angles increase clockwise. In our discrete domain, 12 o'clock is 0, 3 o'clock is ringSize/4, and so on.

When a node is added to the consistent hash, its location is found by applying a hash function like hashItem as described above, with nslots=ringSize. The nodes are stored using a pair of data structures, as follows; this example uses the approximate locations of the nodes N1 through N5 in the circle diagram above (assume ringSize=1024 here):

The positions of the nodes on the circle are stored in slots, which is sorted. nodes holds the corresponding node names. For each i, nodes[i] is at position slots[i] on the circle.

Here's the ConsistentHasher data structure in Go:

type ConsistentHasher struct {
  // nodes is a list of nodes in the hash ring; it's sorted in the same order
  // as slots: for each i, the node at index slots[i] is nodes[i].
  nodes []string

  // slots is a sorted slice of node indices.
  slots []uint64

  ringSize uint64
}

// NewConsistentHasher creates a new consistent hasher with a given maximal
// ring size.
func NewConsistentHasher(ringSize uint64) *ConsistentHasher {
  return &ConsistentHasher{
    ringSize: ringSize,
  }
}

And this is how finding which node a given item maps to is implemented:

// FindNodeFor finds the node an item hashes to. It's an error to call this
// method if the hasher doesn't have any nodes.
func (ch *ConsistentHasher) FindNodeFor(item string) string {
  if len(ch.nodes) == 0 {
    panic("FindNodeFor called when ConsistentHasher has no nodes")
  }
  ih := hashItem(item, ch.ringSize)

  // Since ch.slots is a sorted list of all the node indices for our nodes, a
  // binary search is what we need here. ih is mapped to the node that has the
  // same or the next larger node index. slices.BinarySearch does exactly this,
  // by returning the index where the value would be inserted.
  slotIndex, _ := slices.BinarySearch(ch.slots, ih)

  // When the returned index is len(slots), it means the search wrapped
  // around.
  if slotIndex == len(ch.slots) {
    slotIndex = 0
  }

  return ch.nodes[slotIndex]
}

The key here is the binary search invocation. Adding and removing nodes is done similarly using binary search - see the full code.

Better item distribution with virtual nodes

A common issue that comes up in the implementation of consistent hashing is unbalanced distribution of items across the different nodes. With M items and a total of N nodes, the average distribution will be about \frac{M}{N} per node, but in practice it won't be very balanced - some nodes will have many more items assigned to them than others (see the Appendix for more details).

In a real application, this may mean that some cache servers will be much busier than others, which is a bad thing as far as capacity planning and efficient use of HW. Luckily, there's an elegant tweak to the consistent hashing algorithm that significantly mitigates the problem: virtual nodes.

Instead of mapping each node to a single location on the circle, we'll map it to V locations instead. There are several ways to do this - the simplest is just to tweak the node name in some way. For example, when AddNode is called to add node, it will run:

for i := range V {
  vnodeName = fmt.Sprintf("%v@%v", node, i)

  // ... now add vnodeName to the nodes/slots slices
}

Then, when looking up an item we'll run into one of the virtual nodes, decode the node's name from it (in our example just strip the @<number> suffix) and return that. Implementing node removal is similarly simple.

The idea is that given a node named foo, the virtual node names foo@0, foo@1, foo@2 etc. will be spread all around the circle and not cluster in a single place. See the Appendix for a calculation of how this affects the final distribution.

The source code for this post includes a ConsistentHasherV type that is very similar to ConsistentHasher, except that it implements the virtual node strategy. The user interface remains exactly the same - it's only the internal implementation that changes slightly.

Code

The full source code for this post is on GitHub.

Appendix

The quality of the hash function is very important for good shuffling of nodes on the circle, but even if we take a perfect hash function that produces uniformly distributed values, the outcome is likely to be sub-optimal for our needs.

Let's say we select N points on the unit circle, uniformly in the range [0, 1). If we sort the points by angle, the gaps between neighboring angles are the order statistics. These follow the Beta distribution with parameters (1, N-1), which has a mean of \frac{1}{N} and a variance of \frac{N-1}{N^2(N+1)}.

This is quite significant. Consider a circle with 20 nodes. The standard deviation of the distribution is the square root of the variance; substituting N=20, we get:

\[\sigma=\sqrt{\frac{19}{20^2\cdot 21}}=0.048\]

With 20 nodes uniformly distributed around a circle, we can expect an average of 18 degrees distance between two nodes. A standard deviation of 0.048 means 17 degrees, which is comparable to the average!

We can also do a realistic example to demonstrate this. Let's generate 20 random angles on a circle, and show how the node distribution looks:

In this particular sample, the average angle between two adjacent nodes is 18 degrees (as expected). The smallest angle is just 1.04 degrees, while the largest one is 42 degrees. This means that some nodes will get 40x as many items assigned to them as others!

It's easy to see how virtual nodes help; imagine that each server maps to some number of randomly distributed nodes on the circle; some of these will be farther than others from their closest neighbor, but the average will have much less variety. Mathematically, given a set of n uniformly distributed random variables with variance v, the variance of their average is \frac{v}{n}.

As a concrete experiment, I ran a similar simulation to the one above, but with 10 virtual nodes per node. We'll consider the total portion of the circle mapped to a node when it maps to either of its virtual nodes. While the average remains 18 degrees, the variance is reduced drastically - the smallest one is 11 degrees and the largest 26.

You can find the code for these experiments in the demo.go file of the source code repository.

[1]

This is close to the original motivation for the development of consistent caching by researchers at MIT. Their work, described in the paper "Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web" became the foundation of Akamai Technologies.

There are other use cases of consistent hashing in distributed systems. AWS popularized one common application in their paper "Dynamo: Amazon’s Highly Available Key-value Store", where the algorithm is used to distribute storage keys across servers.

[2] In a real application, we'd probably want to find a different hash function. md5 is relatively slow, and we don't need any of its cryptographic guarantees in this use case. That said, the chosen hash function should be suitable and produce a good mixing of bits even for items that are only slightly different (e.g. "node-1" vs. "node-2"); I found that Go's built-in hash/fnv package isn't great for this purpose, for example.

[3]	A circle is used instead of a linear range to conveniently handle boundary conditions. It's similar to using modular arithmetic for the naive hashing approach.

[4]

I didn't bother to compare performance, but I suspect the array-based approach will be faster for lookup because of being relatively cache-friendly. For insertion/removal, the array approach is O(n) whereas the tree is O(\log\ n), but it's safe to assume that lookup performance is significantly more important. Nodes don't change very often, and lookups happen orders of magnitude more frequently.

Building abstractions using higher-order functions

2023-02-04T05:40:00-08:00

A higher-order function is a function that takes other functions as arguments, or returns a function as its result. Higher-order functions are an exceptionally powerful software design tool because they can easily create new abstractions and are composable. In this post I will present a case study - a set of functions that defines an interesting problem domain. By reading and understanding this code, hopefully anyone can appreciate the power and beauty of higher-order functions and how they enable constructing powerful abstractions from basic building blocks.

One of my all-time favorite programming books is Peter Norvig's PAIP . In section 6.4 - A set of Searching Tools, it presents some code for defining different variants of tree searching that I've always found very elegant.

Here's a quick reimplementation of the main idea in Clojure (see this repository for the full, runnable code); I'm using Clojure since it's a modern Lisp that I enjoy learning and using from time to time.

First, some prerequisites. As is often the case in dynamically-typed Lisp, entities can be described in a very abstract way. The code presented here searches trees, but there is no tree data structure per-se; it's defined using functions. Specifically, there's a notion of a "state" (tree node) and a way to get from a given state to its children states (successors); a function maps between the two.

In our case let's have integers as states; then, an infinite binary tree can be defined using the following successor function:

(defn binary-tree
  "A successors function representing a binary tree."
  [x]
  (list (* 2 x) (+ 1 (* 2 x))))

Given a state (a number), it returns its children as a list. Simplistically, in this tree, node N has the children 2N and 2N+1.

Here are the first few layers of such a tree:

Binary tree with 15 nodes 1-15

In one sense, the tree is infinite because binary-tree will happily return the successors for any node we ask:

paip.core=> (binary-tree 9999)
(19998 19999)

But in another sense, there is no tree. This is a beautiful implication of using functions instead of concrete data - they easily enable lazy evaluation. We cannot materialize an infinite tree inside a necessarily finite computer, but we can operate on it all the same because of this abstraction. As far as the search algorithm is concerned, there exists an abstract state space and we tell it how to navigate and interpret it.

Now we're ready to look at the generic search function:

(defn tree-search
  "Finds a state that satisfies goal?-fn; Starts with states, and searches
  according to successors and combiner. If successful, returns the state;
  otherwise returns nil."
  [states goal?-fn successors combiner]
  (cond (empty? states) nil
        (goal?-fn (first states)) (first states)
        :else (tree-search (combiner (successors (first states))
                                     (rest states))
                           goal?-fn
                           successors
                           combiner)))

Let's dig in. The function accepts the following parameters:

states: a list of starting states for the search. When invoked by the user, this list will typically have a single element; when tree-search calls itself, this list is the states that it plans to explore next.
goal?-fn: a goal detection function. The search doesn't know anything about states and what the goal of the search is, so this is parameterized by a function. goal?-fn is expected to return true for a goal state (the state we were searching for) and false for all other states.
successors: the search function also doesn't know anything about what kind of tree it's searching through; what are the children of a given state? Is it searching a binary tree? A N-nary tree? Something more exotic? All of this is parameterized via the successors function provided by the user.
combiner: finally, the search strategy can be parameterized as well. There are many different kinds of searches possible - BFS, DFS and others. combiner takes a list of successors for the current state the search is looking at, as well as a list of all the other states the search still plans to look at. It combines these into a single list somehow, and thus guides the order in which the search happens.

Even before we see how this function is used, it's already apparent that this is quite a powerful abstraction. tree-search defines the essence of what it means to "search a tree", while being oblivious to what the tree contains, how it's structured and even what order it should be searched in; all of this is supplied by functions passed in as parameters.

Let's see an example, doing a BFS search on our infinite binary tree. First, we define a breadth-first-search function:

(defn breadth-first-search
  "Search old states first until goal is reached."
  [start goal?-fn successors]
  (tree-search (list start) goal?-fn successors prepend))

This function takes a start state (a single state, not a list), goal?-fn and successors, but it sets the combiner parameter to the prepend function, which is defined as follows:

(defn prepend
  [x y]
  (concat y x))

It defines the search strategy (BFS = first look at the rest of the states and only then at successors of the current state), but still leaves the tree structure and the notion of what a goal is to parameters. Let's see it in action:

paip.core=> (breadth-first-search 1 #(= % 9) binary-tree)
9

Here we pass the anonymous function literal #(= % 9) as the goal?-fn parameter. This function simply checks whether the state passed to it is the number 9. We also pass binary-tree as the successors, since we're going to be searching in our infinite binary tree. BFS works layer by layer, so it has no issue with that and finds the state quickly.

We can turn on verbosity (refer to the full code to see how it works) to see what states parameter tree-search gets called with, observing the progression of the search:

paip.core=> (with-verbose (breadth-first-search 1 #(= % 9) binary-tree))
;; Search: (1)
;; Search: (2 3)
;; Search: (3 4 5)
;; Search: (4 5 6 7)
;; Search: (5 6 7 8 9)
;; Search: (6 7 8 9 10 11)
;; Search: (7 8 9 10 11 12 13)
;; Search: (8 9 10 11 12 13 14 15)
;; Search: (9 10 11 12 13 14 15 16 17)
9

This is the prepend combiner in action; for example, after (3 4 5), the combiner prepends (4 5) to the successors of 3 (the list (6 7)), getting (4 5 6 7) as the set of states to search through. Overall, observing the first element in the states list through the printed lines, it's clear this is classical BFS where the tree is visited in "layers".

Implementing DFS using tree-search is similarly easy:

(defn depth-first-search
  "Search new states first until goal is reached."
  [start goal?-fn successors]
  (tree-search (list start) goal?-fn successors concat))

The only difference from BFS is the combiner parameter - here we use concat since we want to examine the successors of the first state before we examine the other states on the list. If we run depth-first-search on our infinite binary tree we'll get a stack overflow (unless we're looking for a state that's on the left-most path), so let's create a safer tree first. This function can serve as a successors to define a "finite" binary tree, with the given maximal state value:

(defn finite-binary-tree
  "Returns a successor function that generates a binary tree with n nodes."
  [n]
  (fn [x]
    (filter #(<= % n) (binary-tree x))))

Note the clever use of higher-order functions here. finite-binary-tree is not a successors function itself - rather it's a generator of such functions; given a value, it creates a new function that acts as successors but limits the the states' value to n.

For example, (finite-binary-tree 15) will create a successors function that represents exactly the binary tree on the diagram above; if we ask it about successors of states on the fourth layer, it will say there are none:

paip.core=> (def f15 (finite-binary-tree 15))
#'paip.core/f15
paip.core=> (f15 4)
(8 9)
paip.core=> (f15 8)
()
paip.core=> (f15 7)
(14 15)
paip.core=> (f15 15)
()

As another test, let's try to look for a state that's not in our finite tree. Out infinite tree theoretically has all the states:

paip.core=> (breadth-first-search 1 #(= % 33) binary-tree)
33

But not the finite tree:

paip.core=> (breadth-first-search 1 #(= % 33) (finite-binary-tree 15))
nil

With our finite tree, we are ready to use depth-first-search:

paip.core=> (with-verbose (depth-first-search 1 #(= % 9) (finite-binary-tree 15)))
;; Search: (1)
;; Search: (2 3)
;; Search: (4 5 3)
;; Search: (8 9 5 3)
;; Search: (9 5 3)
9

Note the search order; when (2 3) is explored, 2's successors (4 5) then come before 3 in the next call; this is the definition of DFS.

We can implement more advanced search strategies using this infrastructure. For example, suppose we have a heuristic that tells us which states to prioritize in order to get to the goal faster (akin to A* search on graphs). We can define a best-first-search that sorts the states according to our heuristic and tries the most promising states first ("best" as in "best looking among the current candidates", not as in "globally best").

First, let's define a couple of helper higher-order functions:

(defn diff
  "Given n, returns a function that computes the distance of its argument from n."
  [n]
  (fn [x] (Math/abs (- x n))))

(defn sorter
  "Returns a combiner function that sorts according to cost-fn."
  [cost-fn]
  (fn [new old]
    (sort-by cost-fn (concat new old))))

diff is a function generator like finite-binary-tree; it takes a target number n and returns a function that computes its parameter x's distance from n.

sorter returns a function that serves as the combiner for our search, based on a cost function. This is done by concatenating the two lists (successors of first state and the rest of the states) first, and then sorting them by the cost function. sorter is a powerful example of modeling with higher-order functions.

With these building blocks in place, we can define best-first-search:

(defn best-first-search
  "Search lowest cost states first until goal is reached."
  [start goal?-fn successors cost-fn]
  (tree-search (list start) goal?-fn successors (sorter cost-fn)))

Once again, this is just like the earlier BFS and DFS - only the strategy (combiner) changes. Let's use it to find 9 again:

paip.core=> (with-verbose (best-first-search 1 #(= % 9) (finite-binary-tree 15) (diff 9)))
;; Search: (1)
;; Search: (3 2)
;; Search: (7 6 2)
;; Search: (6 14 15 2)
;; Search: (12 13 14 15 2)
;; Search: (13 14 15 2)
;; Search: (14 15 2)
;; Search: (15 2)
;; Search: (2)
;; Search: (5 4)
;; Search: (10 11 4)
;; Search: (11 4)
;; Search: (4)
;; Search: (9 8)
9

While it finds the state eventually, we discover that our heuristic is not a great match for this problem, as it sends the search astray. The goal of this post is to demonstrate the power of higher-order functions in building modular code, not to discover an optimal heuristic for searching in binary trees, though :-)

One last search variant before we're ready to wrap up. As we've seen with the infinite tree, sometimes the search space is too large and we have to compromise on which states to look at and which to ignore. This technique works particularly well if the target is not some single value that we must find, but rather we want to get a "good enough" result in a sea of bad options. We can use a technique called beam search; think of a beam of light a flashlight produces in a very dark room; we can see what the beam points at, but not much else.

Beam search is somewhat similar to our best-first-search, but after combining and sorting the list of states to explore, it only keeps the first N, where N is given by the beam-width parameter:

(defn beam-search
  "Search highest scoring states first until goal is reached, but never consider
  more than beam-width states at a time."
  [start goal?-fn successors cost-fn beam-width]
  (tree-search (list start) goal?-fn successors
               (fn [old new]
                 (let [sorted ((sorter cost-fn) old new)]
                   (take beam-width sorted)))))

Once again, higher-order functions at play: as its combiner, beam-search creates an anonymous function that sorts the list based on cost-fn, and then keeps only the first beam-width states on that list.

Exercise: Try to run it - what beam width do you need to set in order to successfully find 9 using our cost heuristic? How can this be improved?

Conclusion

This post attempts a code-walkthrough approach to demonstrating the power of higher-order functions. I always found this particular example from PAIP very elegant; a particularly powerful insight is the distilled difference between DFS and BFS. While most programmers intuitively understand the difference and could write down the pseudo-code for both search strategies, modeling the problem with higher-order functions lets us really get to the essence of the difference - concat vs. prepend as the combiner step.

Why coding interviews aren't all that bad

2022-03-19T07:07:00-07:00

Coding interviews have never been popular in the programming community; I mean, they are prevalent, since many companies still use them to filter candidates, but they are vastly unpopular in the community because people find them too hard, too unfair, too unrepresentative of reality and so on. There are viral stories all around - like when the creator of Homebrew failed Google's interviews [1].

In this post I want to make the case that coding interviews aren't all that bad. They're certainly not a perfect way to filter candidates to SWE positions, but they are among the best tools we've got. I've been interviewing folks for SWE positions for almost 20 years now, and I regularly review hiring packets in which all of the candidate's interviews are laid out with their results and recommendations.

First, what do I mean by "coding interviews"? Typically, writing some moderate amount of code (in the order of 50 lines) throughout the interview, involving nothing more than simple and fundamental data structures. Linked lists, graphs, arrays, binary search trees at most. Many interviews will also examine one or more horizontal aspects like recursion, parsing simple data, and discussing the runtime and storage complexity and performance of simple algorithms.

To reiterate, the best coding interviews are relatively simple questions that don't require deep familiarity with advanced data structres like skip-lists or quad-trees, but do require a good understanding of programming fundamentals. I don't ask candidates to invert binary trees, but whoever does ask this question most likely doesn't ask it because it is relevant to their work projects; rather, they ask it because it probes the candidate's understanding of recursion, basic data structures, debugging and careful treatment of pointers or references (depending on the language).

Objectivity

The reason I prefer asking coding questions is because I firmly believe they are the most objective way to evaluate candidates. Solving a coding question removes so many subjective factors, especially related to the candidate's demographics and background, and the majority of the common interview biases.

A good way to think about this is to consider one of the alternatives commonly proposed to coding interviews: just talk to the candidate, ask about their background, their past job, the problems they were solving, etc. I actually have an educational story related to this, from my own experience.

Many years ago in a company far away, I was just starting to interview full-time candidates (before that I only had experience interviewing interns). In that company, we would do 90-minute interviews in pairs, and I was paired with an experienced engineer. The candidate was amazing at talking about themselves and after 20 minutes I was convinced they should be accepted; my experienced partner, however, calmly proceeded to asking a relatively simple technical question (this was a HW design interview and the question had to do with designing a basic low-pass filter). The candidate was immediately lost and fumbled for an hour. I remember feeling shocked; how is it possible that someone obviously so good can't do something so basic?

I keep encountering a variation of this scenario all the time, to this day, but I'm rarely surprised any more. I learned that just talking to candidates is a sure way to get an extremely biased view of their skills. I do think that in a slate of 4-5 interviews it's important for one of them to be more personal where the interviewer gets to know the candidate and tries to assess if they are pleasant to work with. But the majority of the interviews have to objective and technical, IMO.

Take-home projects

Another commonly mentioned alternative to coding interviews is take-home projects, where the candidates get a sizable assignment to complete at home; this assignment can take on the order of 2-20 hours and is meant to evaluate the candidate on a much more realistic project than a 45-min coding interview.

There are at least three major challenges with this approach.

First of all, candidates don't like these - since they may take a long time to finish, and this doesn't scale well when you're applying to multiple jobs. Many strong candidates will be put off by the requirement to spend multiple hours on a programming assignment; since finding strong candidates is one of the biggest challenges companies face, this is an important factor.

Second, these assignments are a burden on the hiring team as well, since they take a long time to prepare and evaluate. While this may not be a problem for "one off" hiring quests, anyone who's familiar with the plight of SWEs who have to spend lots of time on interviews and hiring while also doing their daily jobs because the company is in a growth spurt, will recognize this issue.

Third and most important - these assignments are very vulnerable to cheating, and cheating is absolutely rampant in the industry. These days it's easy to encounter cheating even on mainstream, open platforms like Reddit, not to mention dedicated services like LeetCode or "code for hire" services where candidates can buy solutions for money. This isn't a new problem, either. When I was toying with RentACoder 15 years ago the majority of projects I ended up doing were either homework or "do my work for me" assignments.

Now, coding questions are vulnerable to a kind of "cheating" as well - for example, you can probably find 95% of the questions Google asks on LeetCode these days, but this is very different IMHO. Sure, you can engorge yourself by reading hundreds of questions & answers on LeetCode, and come to the interview ready. But in my experience, when you're faced with an interview you're still on your own and the interviewer can typically ask tangential questions that will expose someone who just memorized the answers. Enforcement is much harder with take-home exercises.

Diversity

Lately I've seen a lot of discussion about how the way interviews are currently done is bad for diversity. The issue of diversity is very important for the industry, for sure. I agree that companies should be seeking ways to diversify their work force - and how to do this properly is a genuinely hard question that is an active area of academic research.

That said, IMHO coding interviews are a significantly more objective way to evaluate candidates from the diversity standpoint. The "tell me about yourself" style of interviews is extremely subjective and open to bias. What better way to activate the automatic rapport people feel for someone with background similar to theirs, and the ingrained lizard-brain antagonism to anyone different?

"Take-home projects" are much more subjective too, because guess what - the cheating is much more available for someone with the money already. As recent exposés demonstrate, when important life-long goals are in play, people will do whatever in their power to get ahead. Folks from a poor background looking for their first high-paying job will not be in a position to pop a $1,000 for some off-shore programmer to solve their take-home question, but someone else would.

Conclusion

Hopefully this post clarifies why I think that coding interviews, while not perfect, are an important objective evaluation technique for hiring SWEs. I'm not saying that coding interviews should be the only criterion, only that they are an important signal that should not be dropped from any hiring process.

To paraphrase a well-worn Churchill quote:

Indeed it has been said that a coding interview is the worst form of interview except for all those other forms that have been tried from time to time

All of this is IMHO, of course, and I only speak for myself here.

Update (2022-03-29): shortly after this post went out, MIT published an article on reinstating standardized scores (SAT/ACT) as one of criteria for admission. I think it's a very well-written, evidence-driven and thoughtful article. They explicitly state that such tests, while not perfect, provide a more objective criteria for under-represented groups' admissions than other criteria in use. They also provide evidence on how these tests are predictive of students' success in MIT. I found it to be a very nice parallel for this post.

[1] Clearly, this could be a problem in the process but (1) we don't know the full details of the case here and (2) this is a very rare occurrence compared to the tens of thousands of SWE interviews that are conducted every day. The odds of J-random-SWE-candidate being the primary author of well-known SW are extremely low, and the vast majority of these candidates will easily solve typical coding questions. The remaining probability of P(fail | eminent programmer) is negligible.

Building binary trees from inorder-depth lists

2022-02-05T06:33:00-08:00

I ran into an interesting algorithm while hacking on Advent of Code a while ago. This post is a summary of what I've learned.

Consider a binary tree that represents nested pairs, where each pair consists of two kinds of elements: a number, or another pair. For example, the nested pair ((6 9) ((3 4) 2)) is represented with this tree [1]:

(ignore the numbered lines on the right for now, we'll get to them shortly)

Trees representing such pairs have the following characteristics:

Leaves hold numbers, while internal nodes don't hold numbers, but only pointers to child nodes.
Each node in the tree has either 0 or 2 children.
A non-empty tree has at least one internal node.

While looking for alternative solutions to a problem, I ran across Tim Visée's Rust solution which uses an interesting representation of this tree. It's represented by an in-order traversal of the tree, with a list of (value depth) pairs where value is a leaf value and depth is its depth in the tree. The depth starts from 0 at the root - this is what the numbered lines in the diagram above represent.

For our sample tree, the inorder-depth representation is as follows:

(6 2) (9 2) (3 3) (4 3) (2 2)

The surprising realization (at least for me) is that the original tree can be reconstructed from this representation! Note that it's just a list of leaf values - the internal nodes are not specified. It's well known that we can't reconstruct a tree just from its in-order traversal, but a combination of the added depth markers and the restrictions on the tree make it possible.

I'll present a recursive algorithm to reconstruct the tree (based on Tim Visée's code, which does not explicitly rebuild the tree but computes something on it); this algorithm is very clever and isn't easy to grok. Then, I'll present an iterative algorithm which IMHO is easier to understand and explain.

But first, let's start with the data structures. The full (Go) code is available on GitHub.

type DItem struct {
  Value int
  Depth int
}

type DList []DItem

This is our representation of the inorder-depth list - a slice of DItem values, each of which has a numeric value and depth.

The tree itself is just what you'd expect in Go:

type Tree struct {
  Value       int
  Left, Right *Tree
}

Recursive algorithm

Here is the recursive version of the tree reconstruction algorithm:

func (dl DList) BuildTreeRec() *Tree {
  cursor := 0

  var builder func(depth int) *Tree
  builder = func(depth int) *Tree {
    if cursor >= len(dl) {
      return nil
    }

    var left *Tree
    if dl[cursor].Depth == depth {
      left = &Tree{Value: dl[cursor].Value}
      cursor++
    } else {
      left = builder(depth + 1)
    }

    var right *Tree
    if dl[cursor].Depth == depth {
      right = &Tree{Value: dl[cursor].Value}
      cursor++
    } else {
      right = builder(depth + 1)
    }
    return &Tree{Left: left, Right: right}
  }

  return builder(1)
}

I find this algorithm fairly tricky to understand; the combination of double recursion with mutable state is powerful. Some tips:

cursor represents the next item in the inorder-depth list; it may help thinking of it as a queue; taking dl[cursor] and advancing cursor is akin to popping from the head of the queue.
The depth parameter represents the depth in the tree the builder is currently on. If the next item in the queue has a matching depth, we construct a leaf from it. Otherwise, we recurse with higher depth to construct an internal node starting from it.
The basic recursive invariant for builder is: the remaining items in dl represent a pair: build its left side, then build its right side.

If it's still not 100% clear, that's OK. In what follows, I'll describe an alternative formulation of this algorithm - without recursion. IMHO this version is easier to follow, and once one gets it - it's also easier to understand the recursive approach.

Iterative algorithm

To get some intuition for how the algorithm works, let's first work through the example we've using throughout this post. We'll take the inorder-depth representation:

(6 2) (9 2) (3 3) (4 3) (2 2)

And will see how to construct a tree from it, step by step. In what follows, the numbered list walks through inserting the first 6 child nodes into the tree, and the steps correspond one-to-one to the diagrams below the list. Each step of the algorithm inserts one node into the tree (either an internal node or a leaf node with the value). The red "pointer" in the diagrams corresponds to the node inserted by each step.

Let's assume we begin with the root node already created.

To insert (6 2) we have to get to depth 2. The children of the root node would be at depth 1, so we have to create a new internal node first. Since the list is in-order, we create the left child first and move our pointer to it.
Now our current node's children are depth 2, so we can insert (6 2). Since the current node has no left child, we insert 6 as its left child.
The next node to insert is (9 2). The node we've just inserted is a leaf, so we backtrack to its parent. Its children are depth two, and it has no right child, so we insert 9 as its right child.
The next node to insert is (3 3). The current node is a leaf so it can't have children; we climb up to the parent, which already has both its children links created. So we climb up again, to the root. The root has a left child, but no right child. We create the right child.
Since the current node's children are depth 2, we can't insert (3 3) yet. The current node has no left child, so we create it and move into it.
The current node's children are depth 3, so we can insert 3 as its left child.

And so on, until we proceed to insert all the values.

The main thing to notice here is that the insertion follows a strict in-order. We go left as far as possible, then backtrack through the parent and turn right. How much is "possible" is determined by the depth markers in the representation, so there's actually no ambiguity [2].

Before we move on to the code, one important point about reaching a parent from a given node. There are at least two common ways to do this: one is keeping parent links in the nodes, and another is using a stack of parents while constructing the tree. In the code shown below, I opt for the second option - an explicit stack of parent nodes. This code can be easily rewritten with parent links instead (try it as an exercise!)

With all that in place, the code shouldn't be hard to understand; here it is, with copious comments:

// BuildTree builds a Tree from a DList using an iterative algorithm.
func (dl DList) BuildTree() *Tree {
  if len(dl) == 0 {
    return nil
  }
  // result is the tree this function is building. The result pointer always
  // points at the root, so we can return it to the caller. t points to the
  // current node being constructed throughout the algorithm.
  result := &Tree{}
  t := result

  // depth is the current depth of t's children.
  depth := 1

  // stack of parent nodes to implement backtracking up the tree once we're done
  // with a subtree.
  var stack []*Tree

  // The outer loop iterates over all the items in a DList, inserting each one
  // into the tree. Loop invariant: all items preceding this item in dl have
  // already been inserted into the tree, and t points to the node where the
  // last insertion was made.
nextItem:
  for _, item := range dl {
    // The inner loop find the right place for item in the tree and performs
    // insertion.
    // Loop invariant: t points at the node where we're trying to insert, depth
    // is the depth of its children and stack holds a stack of t's parents.
    for {
      // Check if item can be inserted as a child of t; this can be done only if
      // our depth matches the item's and t doesn't have both its children yet.
      // Otherwise, t is not the right place and we have to keep looking.
      if item.Depth == depth && t.Left == nil {
        t.Left = &Tree{Value: item.Value}
        continue nextItem
      } else if item.Depth == depth && t.Right == nil {
        t.Right = &Tree{Value: item.Value}
        continue nextItem
      }

      // We can't insert at t.
      // * If t does not have a left child yet, create it and repeat loop with
      //   this left child as t.
      // * If t does not have a right child yet, create it and repeat loop with
      //   this right child as t.
      // * If t has both children, we have to backtrack up the tree to t's
      //   parent.
      if t.Left == nil {
        stack = append(stack, t)
        t.Left = &Tree{}
        t = t.Left
        depth++
      } else if t.Right == nil {
        stack = append(stack, t)
        t.Right = &Tree{}
        t = t.Right
        depth++
      } else {
        // Pop from the stack to make t point to its parent
        t, stack = stack[len(stack)-1], stack[:len(stack)-1]
        depth--
      }
    }
  }

  return result
}

Final words

If you take some time to convince yourself that the iterative algorithm works, it becomes easier to understand the recursive one... because it's doing the exact same thing! The loops are replaced by recursion; the explicit parent stack is replaced by an implicit call stack of the recursive function, but otherwise - it's the same algorithm [3].

Finally, some credits are due. Thanks to my wife for helping me come up with the iterative formulation of the algorithm. Thanks to Tim Visée for the inspiration for this post.

[1]	Note that this is not a binary search tree; the order of values in the leaves is entirely arbitrary.

[2]	One way the algorithm avoids ambiguity is by requiring that nodes in the tree have either no children or two children. Nodes with one child would confuse the algorithm; can you see why?

[3] Here is an exercise: the code of the iterative algorithm is currently structured to ease understanding, but what happens if we merge the conditions of t.Left == nil, checking it in just one place and then either inserting (if the depth matches) or keep looking; and the same for t.Right. If you make these changes the algorithm will still work (feel free to use the tests in the accompanying code), and it starts resembling the recursive version even more.

Asimov, programming and the meta ladder

2022-01-22T17:02:00-08:00

One of my favorite stories by Isaac Asimov is Profession. The following is a spoiler, so please read the story before proceeding if you don't like spoilers.

In the futuristic society of year 6000-something, people no longer need to learn their profession from books, lectures or hands-on experience. Each person has their brain analyzed at a certain age and then the know-how for the occupation that's best suited for them is simply uploaded into the brain using special cassettes (hey, this story is 60 years old) and electrodes. The folks who end up the best at their craft (determined via competitions) end up with high-demand assignments on "Class A" outer worlds.

The protagonist, George Platen, has a dream of getting "educated" in a certain profession and reaching a desirable assignment. But he runs into trouble when his brain assessment determines that no profession is a good fit for him, and he's placed in a special "house for the feeble-minded" to spend his time however he wants, even reading books.

Long story short, after some adventures George discovers the truth on his own; someone has to create these training cassettes, advance human technology and update training materials to account for these advances. There's a "meta-profession", something akin to scientist, and George was selected for this meta-profession.

I always loved this story for the meta aspect; many occupations are prone to automation, and this has become much more true since Asimov first put the plot to paper. But some human professions are necessarily "meta"; you can automate them, but this just generates new professions that have to develop said automation. Ad infinitum, or at least until Singularity.

In the course of my career, I've heard the promises of "no code" programming many times. These tools didn't cause the demand for programmers to plummet, but to simply shift in other directions. More recently, I treat the hype about AI coding assistants like GitHub Copilot with similar calm. These are great tools that are going to make some programmers' lives easier, but replace programmers? Nope; only move programmers another notch up the meta ladder.

By the way, do you know what profession George Platen was aiming at before he knew the truth? Computer programmer. A far-sighted move by Asimov, given that the story was written in 1957!