Eli Bendersky's website - Functional programming

The power of single-method interfaces in Go

2023-03-17T06:23:00-07:00

The other day I was pondering on the prevalence of single-method interfaces (SMI) in Go, and what makes them so effective and useful. SMIs have proven to be a very successful software modeling tool for Go programmers, and you find them all over Go code-bases.

I tried to think about the fundamentals, which brought me to some of the earliest roots of our trade: functional programming and higher-order functions (HOF). I discussed some examples of applying higher-order functions in Go recently.

This post will describe how SMIs are a more general and powerful technique than HOFs. It makes the following claims:

SMIs can do whatever HOFs can
SMIs are more general
SMIs are somewhat more verbose for simple cases

To begin, let's use the same example as before.

The tree search example using SMIs

The previous post demonstrated a Go solution using higher-order functions for the tree search problem described earlier. I encourage you to review the earlier posts to get the most out of this one.

Let's now see how the same task can be accomplished using SMIs instead of HOFs; the full code is on GitHub. Starting with the types:

type State int
type States []State

// GoalDetector is an interface that wraps a single IsGoal method. IsGoal
// takes a state and determines whether it's a goal state.
type GoalDetector interface {
  IsGoal(s State) bool
}

// SuccessorGenerator is an interface that wraps a single Successors method.
// Successors returns the successors of a state.
type SuccessorGenerator interface {
  Successors(s State) States
}

// Combiner is an interface that wraps a single Combine method. Combine
// determines the search strategy by combining successors of the current state
// with all the other states into a single list of states.
type Combiner interface {
  Combine(succ States, others States) States
}

These are the equivalent SMIs to the GoalP, Successors and Combiner function types we've seen before; the names are slightly modified to be more suitable for interfaces and their methods.

The tree search itself - using these interfaces - is almost identical to the previous version:

func treeSearch(states States, gd GoalDetector, sg SuccessorGenerator, combiner Combiner) State {
  if len(states) == 0 {
    return -1
  }

  first := states[0]
  if gd.IsGoal(first) {
    return first
  } else {
    return treeSearch(combiner.Combine(sg.Successors(first), states[1:]), gd, sg, combiner)
  }
}

To implement BFS, we reuse prependOthers from the previous post (it remains identical):

And again, appendOthers and implementing DFS:

func bfsTreeSearch(start State, gd GoalDetector, sg SuccessorGenerator) State {
  return treeSearch(States{start}, gd, sg, CombineFunc(prependOthers))
}

What is CombineFunc? Since prependOthers is a function - while treeSearch takes interfaces - we need an adapter type:

type CombineFunc func(States, States) States

func (f CombineFunc) Combine(succ States, others States) States {
  return f(succ, others)
}

If you have a bit of Go experience already, this may seem familiar. It's exactly the same breed of adapter as http.HandlerFunc. Unfortunately, at this time a Go function cannot be assigned to a SMI with a compatible method signature, so these adapters are required (see the Appendix for more details).

The rest of the code is also very similar, with an adapter thrown in here and there. Feel free to read the rest of it and compare to the HOF version.

Since a SMI wraps a method, it can do whatever a function type can do. We can pass SMIs to functions; we can return SMIs from functions. There are tons of examples of this across the Go standard library, and in almost any non-trivial Go project.

So is this it? Are SMIs just a cumbersome variation on higher-order functions? No, not at all. The code sample used in this blog post is unfair to SMIs because it was originally written and designed for HOFs. What's important to take from here is that it's also possible to accomplish the same goal with SMIs. This is the basis of the power of SMIs - they can serve the same role as higher-order functions; but they have additional features that HOFs lack.

SMIs are more general than function types

In examining the generality of SMIs, the topic of state immediately comes up. I will only discuss it briefly, because it's a known red herring. One could claim that SMIs are more general because they can represent objects with built-in state (for example, a struct value where the struct type implements a SMI). Functions can represent state too, however, by means of closures. In fact, our sample code clearly shows this in functions like stateIs and costDiffTarget.

The real reasons for why SMIs are more general are:

SMIs naturally extend to multi-method interfaces
Go values can implement more than one interface

Let's discuss these in detail.

Single-method interfaces; it's in the name! Interfaces with a single method to implement, like the ubiquitous io.Reader:

type Reader interface {
  Read(p []byte) (n int, err error)
}

But just as easily, Go has multi-method interfaces, like io.ReadWriter [1]:

type ReadWriter interface {
  Read(p []byte) (n int, err error)
  Write(p []byte) (n int, err error)
}

There are many other useful multi-method interfaces in the standard library: net/http.ResponseWriter, context.Context, fs.File and so on. While the extension from a SMI to a multi-method interface in Go is trivial, this is difficult and unnatural to achieve with function types [2].

Next, a single Go value can implement multiple interfaces. Implicit interface implementation is one of the banner features of Go - a type does not have to declare that it implements an interface; it does so implicitly if it has the right method. There are many good examples of this in the Go standard library; for example the encoding.BinaryMarshaler interface:

type BinaryMarshaler interface {
  MarshalBinary() (data []byte, err error)
}

And the encoding/json.Marshaler interface:

type Marshaler interface {
  MarshalJSON() ([]byte, error)
}

A type like time.Time implements both, and can thus be encoded into a binary format and JSON.

Another interesting example is the fmt.Stringer interface:

type Stringer interface {
  String() string
}

Many Go types implement this interface along with other interfaces (time.Time does too, of course); overall it's not uncommon to see large, sophisticated types implementing 5 or more interfaces.

Both of the capabilities presented here (multi-method interfaces and a single value implementing multiple SMIs) are very hard to achieve with function types and first-class functions.

Conclusion

The goal of this post was to discuss the prevalence and power of single-method interfaces (SMIs) in Go. It started by showing how SMIs can do whatever higher-order functions (HOFs) can do (albeit somewhat more verbosely, given that the task at hand was tailored for HOFs in previous posts).

Then, we discussed additional capabilities of SMIs that make them unique and aren't achievable with HOFs in a reasonable way.

In functional languages, HOFs are heavily used to implement sophisticated and useful abstractions. In Go, you can use HOFs as well but most Go code prefers to use SMIs for this purpose, because of the added generality.

So which approach should you use? The best tool for the job at hand, of course. While SMIs are more general and powerful, they are also more verbose for the simple cases. When all you need is a function - use a function! Otherwise, interfaces are a powerful and versatile tool Go programmers employ for many tasks.

Appendix: adapter types for converting functions to SMIs

There is some work in the Go developer community to reduce the toil required to convert between a plain function and a SMI. Here's one proposal that was discussed fairly recently and is currently at an advanced stage - waiting for a prototype before decision. In our example with the Combiner interface, this would enable writing:

Combiner(prependOthers).Combine

Instead of having to define the CombineFunc adapter. While it's still not as terse as passing prependOthers where a Combiner is expected, it's an appreciable reduction in boilerplate. For the more ambitious notion of assignability, there's an older, farther-reaching proposal.

[1]	This interface is actually defined using interface embedding, but the effect is the same.

[2]

It's theoretically possible using closures that return a dispatch function accepting multiple commands and invoking a different operation for each; this technique is described in chapter 3 of SICP. It's so unlike the typical use of HOFs that I consider it just a (cool) implementation trick, showing how interfaces are implemented using simpler building blocks.

Higher-order functions in Go

2023-03-04T19:37:00-08:00

A few weeks ago I wrote about the powerful abstractions that higher-order functions enable, using Clojure code as an example.

In this post I want to show equivalent code using a modern statically-typed language - Go. Please make sure to review the previous post first, since it describes the problem statement in detail.

Go and higher-order functions

While Go has many of the building blocks required to support functional programming, at this time it's not considered a mainstream programming paradigm for the language.

That said, my favorite part of functional programming is definitely supported and widely used in Go - higher-order functions. Go makes it simple to pass functions as arguments to other functions, and return functions from functions. Moreover, it has first-class support for closures, which makes the function-based design process much more powerful. This also meshes well with Go's automatic memory management, mostly eliminating worries about values going in and out of scope in closures.

The tree search example in Go

Let's reproduce the tree search example from the previous post in Go; the full code is here. The first task we run into is types; since Go is statically typed, we have to define our types explicitly:

type State int
type States []State

// GoalP takes a state and determines whether it's a goal state.
type GoalP func(s State) bool

// Successors returns the successors of a state.
type Successors func(s State) States

// Combiner determines the search strategy by combining successors of the
// current state with all the other states into a single list of states.
type Combiner func(succ States, others States) States

And this is the tree search function:

// treeSearch returns the state if it's found in the tree; returns -1 if such a
// state wasn't found.
func treeSearch(states States, goalp GoalP, succ Successors, combiner Combiner) State {
  if len(states) == 0 {
    return -1
  }

  first := states[0]
  if goalp(first) {
    return first
  } else {
    return treeSearch(combiner(succ(first), states[1:]), goalp, succ, combiner)
  }
}

While it's somewhat more verbose, its logic follows the Clojure code very closely. Now we're ready to define BFS search:

// prependOthers is a Combiner function that prepends others to succ.
func prependOthers(succ States, others States) States {
  return append(others, succ...)
}

func bfsTreeSearch(start State, goalp GoalP, succ Successors) State {
  return treeSearch(States{start}, goalp, succ, prependOthers)
}

To see how this function is invoked, we'll define a general infinite binary tree Successors function, and another for finite binary trees:

func binaryTree(s State) States {
  return []State{s * 2, s*2 + 1}
}

func finiteBinaryTree(n State) Successors {
  return func(s State) States {
    return filter(binaryTree(s), func(item State) bool { return item <= n })
  }
}

finiteBinaryTree is a nice example of higher-order functions in Go. It takes a value and returns a function that adheres to the Successors function type; in fact, it returns a new function created at runtime - a closure that closes over the value n. Moreover, the function it returns also makes use of higher-order functions in its body, because it invokes filter with a filtering function as a parameter. filter is:

// filter filters a slice based on a predicate, returning a new slice whose
// elements fulfill the predicate.
func filter[T any](s []T, pred func(item T) bool) []T {
  var result []T
  for _, item := range s {
    if pred(item) {
      result = append(result, item)
    }
  }
  return result
}

To call bfsTreeSearch we need to figure out another function parameter - the goal function (GoalP). We can use a higher-order function for this as well:

// stateIs returns a GoalP that checks a state for equality with n.
func stateIs(n State) GoalP {
  return func(s State) bool { return n == s }
}

Now we're ready to invoke bfsTreeSearch:

treeLimit := 30
tree := finiteBinaryTree(State(treeLimit))

bfsFound := bfsTreeSearch(1, stateIs(17), tree)

Note how similar this code is to the Clojure variant; Go supports all the same higher-order function constructs, letting us build powerful abstractions using them. The static types make the code somewhat more verbose, but this has advantages as well!

Imagine that you don't remember in which order successors and combiner should be passed to tree-search in the Clojure code; if you pass them in the wrong order, the code will build fine and you will get an error at runtime about invoking some function with the wrong number of arguments. This is lucky, because these two functions just happen to have different arities; in less lucky cases the error could come much later or be more obscure; or, there could be no error at all and the result could be incorrect.

In Go if you pass succ after combiner, you'll just get a compile-time type error.

Types have a benefit for documentation and code-reading purposes as well, but I don't want to turn this post into a static vs. dynamic typing debate. Let's look at some of the more advanced tree searching functions instead.

Even more higher-order functions in best-first search

A slightly more advanced example involves a "best"-first search; a greedy heuristic that sorts all state candidates by some criteria and pursues them in a priority order. We can still reuse our treeSearch function for it, but have to devise an appropriate Combiner. Let's start with a new function type:

type CostFunc func(s State) int

Given a state s, a cost function returns the integer cost. Now we can write our combiner that sorts a list of states based on a cost function:

func sorter(cost CostFunc) Combiner {
  return func(succ States, others States) States {
    all := append(succ, others...)
    sort.Slice(all, func(i, j int) bool {
      return cost(all[i]) < cost(all[j])
    })
    return all
  }
}

Another nice example of higher-order functions in action. sorter can be seen as a generator of Combiner functions; it can generate a new Combiner given a cost function. At this point we have all we need to implement bestCostTreeSearch:

func bestCostTreeSearch(start State, goalp GoalP, succ Successors, cost CostFunc) State {
  return treeSearch(States{start}, goalp, succ, sorter(cost))
}

The only thing remaining is figuring out what cost function we'd like to pass to the search. A simple one just measures the linear distance of a given state from our goal state:

// costDiffTarget creates a cost function that uses numerical distance from `n`
// as the cost.
func costDiffTarget(n State) CostFunc {
  return func(s State) int {
    delta := int(s) - int(n)
    if delta < 0 {
      return -delta
    } else {
      return delta
    }
  }
}

Given a target state, costDiffTarget generates a function that can serve as the cost function for search. We can now invoke bestCostTreeSearch:

treeLimit := 30
tree := finiteBinaryTree(State(treeLimit))

bestFound := bestCostTreeSearch(1, stateIs(17), tree, costDiffTarget(17))

Conclusion

Go has complete and powerful support for higher-order functions - including proper closures. While the syntax - mostly due to static typing - is somewhat more verbose than in other languages, it's more of an issue for code samples than it is for large code bases.

The verbosity issue is also in discussion by the Go developers, with proposals like Lightweight anonymous function syntax promising terser syntax and more powerful type inference. So there's hope for further improvement in this area.

Go itself uses higher-order functions extensively in the standard library; some examples include bufio.Scanner.Split, a whole array of *Func functions in the strings package (e.g. FieldsFunc), the sort.Slice function used in this post and of course net/http.ServeMux.HandleFunc.

Examples of returning functions also abound; the best known is probably the convention of the context package to return context.CancelFunc for context cancellation. But there are other examples, like net/http.ProxyUrl.

Additional resources, if you found this post interesting:

A followup on The power of single-method interfaces in Go.
The Y combinator in Go with generics for a more mind-bending example of Go's higher-order functions.

Building abstractions using higher-order functions

2023-02-04T05:40:00-08:00

A higher-order function is a function that takes other functions as arguments, or returns a function as its result. Higher-order functions are an exceptionally powerful software design tool because they can easily create new abstractions and are composable. In this post I will present a case study - a set of functions that defines an interesting problem domain. By reading and understanding this code, hopefully anyone can appreciate the power and beauty of higher-order functions and how they enable constructing powerful abstractions from basic building blocks.

One of my all-time favorite programming books is Peter Norvig's PAIP . In section 6.4 - A set of Searching Tools, it presents some code for defining different variants of tree searching that I've always found very elegant.

Here's a quick reimplementation of the main idea in Clojure (see this repository for the full, runnable code); I'm using Clojure since it's a modern Lisp that I enjoy learning and using from time to time.

First, some prerequisites. As is often the case in dynamically-typed Lisp, entities can be described in a very abstract way. The code presented here searches trees, but there is no tree data structure per-se; it's defined using functions. Specifically, there's a notion of a "state" (tree node) and a way to get from a given state to its children states (successors); a function maps between the two.

In our case let's have integers as states; then, an infinite binary tree can be defined using the following successor function:

(defn binary-tree
  "A successors function representing a binary tree."
  [x]
  (list (* 2 x) (+ 1 (* 2 x))))

Given a state (a number), it returns its children as a list. Simplistically, in this tree, node N has the children 2N and 2N+1.

Here are the first few layers of such a tree:

Binary tree with 15 nodes 1-15

In one sense, the tree is infinite because binary-tree will happily return the successors for any node we ask:

paip.core=> (binary-tree 9999)
(19998 19999)

But in another sense, there is no tree. This is a beautiful implication of using functions instead of concrete data - they easily enable lazy evaluation. We cannot materialize an infinite tree inside a necessarily finite computer, but we can operate on it all the same because of this abstraction. As far as the search algorithm is concerned, there exists an abstract state space and we tell it how to navigate and interpret it.

Now we're ready to look at the generic search function:

(defn tree-search
  "Finds a state that satisfies goal?-fn; Starts with states, and searches
  according to successors and combiner. If successful, returns the state;
  otherwise returns nil."
  [states goal?-fn successors combiner]
  (cond (empty? states) nil
        (goal?-fn (first states)) (first states)
        :else (tree-search (combiner (successors (first states))
                                     (rest states))
                           goal?-fn
                           successors
                           combiner)))

Let's dig in. The function accepts the following parameters:

states: a list of starting states for the search. When invoked by the user, this list will typically have a single element; when tree-search calls itself, this list is the states that it plans to explore next.
goal?-fn: a goal detection function. The search doesn't know anything about states and what the goal of the search is, so this is parameterized by a function. goal?-fn is expected to return true for a goal state (the state we were searching for) and false for all other states.
successors: the search function also doesn't know anything about what kind of tree it's searching through; what are the children of a given state? Is it searching a binary tree? A N-nary tree? Something more exotic? All of this is parameterized via the successors function provided by the user.
combiner: finally, the search strategy can be parameterized as well. There are many different kinds of searches possible - BFS, DFS and others. combiner takes a list of successors for the current state the search is looking at, as well as a list of all the other states the search still plans to look at. It combines these into a single list somehow, and thus guides the order in which the search happens.

Even before we see how this function is used, it's already apparent that this is quite a powerful abstraction. tree-search defines the essence of what it means to "search a tree", while being oblivious to what the tree contains, how it's structured and even what order it should be searched in; all of this is supplied by functions passed in as parameters.

Let's see an example, doing a BFS search on our infinite binary tree. First, we define a breadth-first-search function:

(defn breadth-first-search
  "Search old states first until goal is reached."
  [start goal?-fn successors]
  (tree-search (list start) goal?-fn successors prepend))

This function takes a start state (a single state, not a list), goal?-fn and successors, but it sets the combiner parameter to the prepend function, which is defined as follows:

(defn prepend
  [x y]
  (concat y x))

It defines the search strategy (BFS = first look at the rest of the states and only then at successors of the current state), but still leaves the tree structure and the notion of what a goal is to parameters. Let's see it in action:

paip.core=> (breadth-first-search 1 #(= % 9) binary-tree)
9

Here we pass the anonymous function literal #(= % 9) as the goal?-fn parameter. This function simply checks whether the state passed to it is the number 9. We also pass binary-tree as the successors, since we're going to be searching in our infinite binary tree. BFS works layer by layer, so it has no issue with that and finds the state quickly.

We can turn on verbosity (refer to the full code to see how it works) to see what states parameter tree-search gets called with, observing the progression of the search:

paip.core=> (with-verbose (breadth-first-search 1 #(= % 9) binary-tree))
;; Search: (1)
;; Search: (2 3)
;; Search: (3 4 5)
;; Search: (4 5 6 7)
;; Search: (5 6 7 8 9)
;; Search: (6 7 8 9 10 11)
;; Search: (7 8 9 10 11 12 13)
;; Search: (8 9 10 11 12 13 14 15)
;; Search: (9 10 11 12 13 14 15 16 17)
9

This is the prepend combiner in action; for example, after (3 4 5), the combiner prepends (4 5) to the successors of 3 (the list (6 7)), getting (4 5 6 7) as the set of states to search through. Overall, observing the first element in the states list through the printed lines, it's clear this is classical BFS where the tree is visited in "layers".

Implementing DFS using tree-search is similarly easy:

(defn depth-first-search
  "Search new states first until goal is reached."
  [start goal?-fn successors]
  (tree-search (list start) goal?-fn successors concat))

The only difference from BFS is the combiner parameter - here we use concat since we want to examine the successors of the first state before we examine the other states on the list. If we run depth-first-search on our infinite binary tree we'll get a stack overflow (unless we're looking for a state that's on the left-most path), so let's create a safer tree first. This function can serve as a successors to define a "finite" binary tree, with the given maximal state value:

(defn finite-binary-tree
  "Returns a successor function that generates a binary tree with n nodes."
  [n]
  (fn [x]
    (filter #(<= % n) (binary-tree x))))

Note the clever use of higher-order functions here. finite-binary-tree is not a successors function itself - rather it's a generator of such functions; given a value, it creates a new function that acts as successors but limits the the states' value to n.

For example, (finite-binary-tree 15) will create a successors function that represents exactly the binary tree on the diagram above; if we ask it about successors of states on the fourth layer, it will say there are none:

paip.core=> (def f15 (finite-binary-tree 15))
#'paip.core/f15
paip.core=> (f15 4)
(8 9)
paip.core=> (f15 8)
()
paip.core=> (f15 7)
(14 15)
paip.core=> (f15 15)
()

As another test, let's try to look for a state that's not in our finite tree. Out infinite tree theoretically has all the states:

paip.core=> (breadth-first-search 1 #(= % 33) binary-tree)
33

But not the finite tree:

paip.core=> (breadth-first-search 1 #(= % 33) (finite-binary-tree 15))
nil

With our finite tree, we are ready to use depth-first-search:

paip.core=> (with-verbose (depth-first-search 1 #(= % 9) (finite-binary-tree 15)))
;; Search: (1)
;; Search: (2 3)
;; Search: (4 5 3)
;; Search: (8 9 5 3)
;; Search: (9 5 3)
9

Note the search order; when (2 3) is explored, 2's successors (4 5) then come before 3 in the next call; this is the definition of DFS.

We can implement more advanced search strategies using this infrastructure. For example, suppose we have a heuristic that tells us which states to prioritize in order to get to the goal faster (akin to A* search on graphs). We can define a best-first-search that sorts the states according to our heuristic and tries the most promising states first ("best" as in "best looking among the current candidates", not as in "globally best").

First, let's define a couple of helper higher-order functions:

(defn diff
  "Given n, returns a function that computes the distance of its argument from n."
  [n]
  (fn [x] (Math/abs (- x n))))

(defn sorter
  "Returns a combiner function that sorts according to cost-fn."
  [cost-fn]
  (fn [new old]
    (sort-by cost-fn (concat new old))))

diff is a function generator like finite-binary-tree; it takes a target number n and returns a function that computes its parameter x's distance from n.

sorter returns a function that serves as the combiner for our search, based on a cost function. This is done by concatenating the two lists (successors of first state and the rest of the states) first, and then sorting them by the cost function. sorter is a powerful example of modeling with higher-order functions.

With these building blocks in place, we can define best-first-search:

(defn best-first-search
  "Search lowest cost states first until goal is reached."
  [start goal?-fn successors cost-fn]
  (tree-search (list start) goal?-fn successors (sorter cost-fn)))

Once again, this is just like the earlier BFS and DFS - only the strategy (combiner) changes. Let's use it to find 9 again:

paip.core=> (with-verbose (best-first-search 1 #(= % 9) (finite-binary-tree 15) (diff 9)))
;; Search: (1)
;; Search: (3 2)
;; Search: (7 6 2)
;; Search: (6 14 15 2)
;; Search: (12 13 14 15 2)
;; Search: (13 14 15 2)
;; Search: (14 15 2)
;; Search: (15 2)
;; Search: (2)
;; Search: (5 4)
;; Search: (10 11 4)
;; Search: (11 4)
;; Search: (4)
;; Search: (9 8)
9

While it finds the state eventually, we discover that our heuristic is not a great match for this problem, as it sends the search astray. The goal of this post is to demonstrate the power of higher-order functions in building modular code, not to discover an optimal heuristic for searching in binary trees, though :-)

One last search variant before we're ready to wrap up. As we've seen with the infinite tree, sometimes the search space is too large and we have to compromise on which states to look at and which to ignore. This technique works particularly well if the target is not some single value that we must find, but rather we want to get a "good enough" result in a sea of bad options. We can use a technique called beam search; think of a beam of light a flashlight produces in a very dark room; we can see what the beam points at, but not much else.

Beam search is somewhat similar to our best-first-search, but after combining and sorting the list of states to explore, it only keeps the first N, where N is given by the beam-width parameter:

(defn beam-search
  "Search highest scoring states first until goal is reached, but never consider
  more than beam-width states at a time."
  [start goal?-fn successors cost-fn beam-width]
  (tree-search (list start) goal?-fn successors
               (fn [old new]
                 (let [sorted ((sorter cost-fn) old new)]
                   (take beam-width sorted)))))

Once again, higher-order functions at play: as its combiner, beam-search creates an anonymous function that sorts the list based on cost-fn, and then keeps only the first beam-width states on that list.

Exercise: Try to run it - what beam width do you need to set in order to successfully find 9 using our cost heuristic? How can this be improved?

Conclusion

This post attempts a code-walkthrough approach to demonstrating the power of higher-order functions. I always found this particular example from PAIP very elegant; a particularly powerful insight is the distilled difference between DFS and BFS. While most programmers intuitively understand the difference and could write down the pseudo-code for both search strategies, modeling the problem with higher-order functions lets us really get to the essence of the difference - concat vs. prepend as the combiner step.

The Y combinator in Go with generics

2022-06-11T07:55:00-07:00

A few years ago I've written in some detail about the Y combinator, including implementations in Clojure and Python.

This short post is showing how to implement the Y combinator in Go using generics; in general, static typing makes the Y combinator somewhat less appealing, but I figured that generics will at least allow the implementation to be reasonably reusable. Not that this is useful in any real-world scenario, of course :-)

I won't explain how the Y combinator itself works here (please refer to the previous post for that), only the Go specifics. The full, runnable code for this post is available on GitHub.

type Func[T, U any] func(T) U
type TagFunc[T, U any] func(Func[T, U]) Func[T, U]
type CombinatorFunc[T, U any] func(CombinatorFunc[T, U]) Func[T, U]

The Y combinator's definition uses several kinds of anonymous functions. In Go, we need those to have explicit types. Func is the underlying computation function - it's what the function type would be if we'd used normal recursion. It's parameterized with two generic types: T for its parameter type and U for its return type.

FuncTag is the type of the functions users should write now. CombinatorFunc is only used in the definition of the Y combinator itself:

func Y[T, U any](f TagFunc[T, U]) Func[T, U] {
  return func(self CombinatorFunc[T, U]) Func[T, U] {
    return f(func(n T) U {
      return self(self)(n)
    })
  }(func(self CombinatorFunc[T, U]) Func[T, U] {
    return f(func(n T) U {
      return self(self)(n)
    })
  })
}

Here's how to use it. To define a recursive factorial-computing function that does not refer to itself by name, the user has to create a "tag" function that takes and returns a Func. At this point we also instantiate the Func to have the exact types we need:

var factorial_tag = func(recurse Func[int, int]) Func[int, int] {
  return func(n int) int {
    if n == 0 {
      return 1
    }
    return n * recurse(n-1)
  }
}

And the actual factorial function that users can call is created with:

fac := Y(factorial_tag)

Now fac is function of type Func: it computes and returns the factorial of its parameter, and can be simply invoked as answer := fac(param).

We can also try the other example in my previous post - a function that finds the sum of values in a binary tree. It demonstrates a slightly more complicated recursive flow as well as different types for the function's parameter and return value:

type Node struct {
  val   int
  left  *Node
  right *Node
}

var treesum_tag = func(recurse Func[*Node, int]) Func[*Node, int] {
  return func(n *Node) int {
    if n == nil {
      return 0
    } else {
      return n.val + recurse(n.left) + recurse(n.right)
    }
  }
}

And once again, the actual usable function is generated by invoking Y:

treesum := Y(treesum_tag)

Type inference

2018-11-14T06:16:00-08:00

Type inference is a major feature of several programming languages, most notably languages from the ML family like Haskell. In this post I want to provide a brief overview of type inference, along with a simple Python implementation for a toy ML-like language.

Uni-directional type inference

While static typing is very useful, one of its potential downsides is verbosity. The programmer has to annotate values with types throughout the code, which results in more effort and clutter. What's really annoying, though, is that in many cases these annotations feel superfluous. Consider this classical C++ example from pre-C++11 times:

std::vector<Blob*> blobs;
std::vector<Blob*>::iterator iter = blobs.begin();

Clearly when the compiler sees blobs.begin(), it knows the type of blobs, so it also knows the type of the begin() method invoked on it because it is familiar with the declaration of begin. Why should the programmer be burdened with spelling out the type of the iterator? Indeed, one of the most welcome changes in C++11 was lifting this burden by repurposing auto for basic type inference:

std::vector<Blob*> blobs;
auto iter = blobs.begin();

Go has a similar capability with the := syntax. Given some function:

func parseThing(...) (Node, error) {
}

We can simply write:

node, err := parseThing(...)

Without having to explicitly declare that node has type Node and err has type error.

These features are certainly useful, and they involve some degree of type inference from the compiler. Some functional programming proponents say this is not real type inference, but I think the difference is just a matter of degree. There's certainly some inference going on here, with the compiler calculating and assigning the right types for expressions without the programmer's help. Since this calculation flows in one direction (from the declaration of the vector::begin method to the auto assignment), I'll call it uni-directional type inference [1].

Bi-directional type inference (Hindley-Milner)

If we define a new map function in Haskell to map a function over a list, we can do it as follows:

mymap f [] = []
mymap f (first:rest) = f first : mymap f rest

Note that we did not specify the types for either the arguments of mymap, or its return value. The Haskell compiler can infer them on its own, using the definition provided:

> :t Main.mymap
Main.mymap :: (t1 -> t) -> [t1] -> [t]

The compiler has determined that the first argument of mymap is a generic function, assigning its argument the type t1 and its return value the type t. The second argument of mymap has the type [t1], which means "list of t1"; then the return value of mymap has the type "list of t". How was this accomplished?

Let's start with the second argument. From the [] = [] variant, and also from the (first:rest) deconstruction, the compiler infers it has a list type. But there's nothing else in the code constraining the element type, so the compiler chooses a generic type specifier - t1. f first applies f to an element of this list, so f has to take t1; nothing constrains its return value type, so it gets the generic t. The result is f has type (t1 -> t), which in Haskell parlance means "a function from t1 to t".

Here is another example, written in a toy language I put together for the sake of this post. The language is called microml, and its implementation is described at the end of the post:

foo f g x = if f(x == 1) then g(x) else 20

Here foo is declared as a function with three arguments. What is its type? Let's try to run type inference manually. First, note that the body of the function consists of an if expresssion. As is common in programming languages, this one has some strict typing rules in microml; namely, the type of the condition is boolean (Bool), and the types of the then and else clauses must match.

So we know that f(x == 1) has to return a Bool. Moreover, since x is compared to an integer, x is an Int. What is the type of g? Well, it has an Int argument, and it return value must match the type of the else clause, which is an Int as well.

To summarize:

The type of x is Int
The type of f is Bool -> Bool
The type of g is Int -> Int

So the overall type of foo is:

((Bool -> Bool), (Int -> Int), Int) -> Int

It takes three arguments, the types of which we have determined, and returns an Int.

Note how this type inference process is not just going in one direction, but seems to be "jumping around" the body of the function figuring out known types due to typing rules. This is why I call it bi-directional type inference, but it's much better known as Hindley-Milner type inference, since it was independently discovered by Roger Hindley in 1969 and Robin Milner in 1978.

How Hindley-Milner type inference works

We've seen a couple of examples of manually running type inference on some code above. Now let's see how to translate it to an implementable algorithm. I'm going to present the process in several separate stages, for simplicity. Some other presentations of the algorithm combine several of these stages, but seeing them separately is more educational, IMHO.

The stages are:

Assign symbolic type names (like t1, t2, ...) to all subexpressions.
Using the language's typing rules, write a list of type equations (or constraints) in terms of these type names.
Solve the list of type equations using unification.

Let's use this example again:

foo f g x = if f(x == 1) then g(x) else 20

Starting with stage 1, we'll list all subexpressions in this declaration (starting with the declaration itself) and assign unique type names to them:

foo                                       t0
f                                         t1
g                                         t2
x                                         t3
if f(x == 1) then g(x) else 20            t4
f(x == 1)                                 t5
x == 1                                    t6
x                                         t3
g(x)                                      t7
20                                        Int

Note that every subexpression gets a type, and we de-duplicate them (e.g. x is encountered twice and gets the same type name assigned). Constant nodes get known types.

In stage 2, we'll use the language's typing rules to write down equations involving these type names. Usually books and papers use slightly scary formal notation for typing rules; for example, for if:

\[\frac{\Gamma \vdash e_0 : Bool, \Gamma \vdash e_1 : T, \Gamma \vdash e_2 : T}{\Gamma \vdash if\: e_0\: then\: e_1\: else\: e_2 : T}\]

All this means is the intuitive typing of if we've described above: the condition is expected to be boolean, and the types of the then and else clauses are expected to match, and their type becomes the type of the whole expression.

To unravel the notation, prepend "given that" to the expression above the line and "we can derive" to the expression below the line; \Gamma \vdash e_0 : Bool means that e_0 is typed to Bool in the set of typing assumptions called \Gamma.

Similarly, a typing rule for single-argument function application would be:

\[\frac{\Gamma \vdash e_0 : T, \Gamma \vdash f : T \rightarrow U}{\Gamma \vdash f(e_0) : U}\]

The real trick of type inference is running these typing rules in reverse. The rule tells us how to assign types to the whole expression given its constituent types, but we can also use it as an equation that works both ways and lets us infer constituent types from the whole expression's type.

Let's see what equations we can come up with, looking at the code:

From f(x == 1) we infer t1 = (t6 -> t5), because t1 is the type of f, t6 is the type of x == 1, and t5 is the type of f(x == 1). Note that we're using the typing rules for function application here. Moreover, we can infer that t3 is Int and t6 is Bool because of the typing rule of the == operator.

Similarly, from g(x) we infer t2 = (t3 -> t7).

From the if expression, we infer that t6 is Bool (since it's the condition of the if) and that t4 = Int, because the then and else clauses must match.

Now we have a list of equations, and our task is to find the most general solution, treating the equations as constraints. This is done by using the unification algorithm which I described in detail in the previous post. The solution we're seeking here is precisely the most general unifier.

For our expression, the algorithm will find the type of foo to be:

((Bool -> Bool), (Int -> Int), Int) -> Int)

As expected.

If we make a slight modification to the expression to remove the comparison of x with 1:

foo f g x = if f(x) then g(x) else 20

Then we can no longer constrain the type of x, since all we know about it is that it's passed into functions f and g, and nothing else constrains the arguments of these functions. The type inference process will thus calculate this type for foo:

((a -> Bool), (a -> Int), a) -> Int

It assigns x the generic type name a, and uses it for the arguments of f and g as well.

The implementation

An implementation of microml is available here, as a self-contained Python program that parses a microml declaration and infers its type. The best starting point is main.py, which spells out the stages of type inference:

code = 'foo f g x = if f(x == 1) then g(x) else 20'
print('Code', '----', code, '', sep='\n')

# Parse the microml code snippet into an AST.
p = parser.Parser()
e = p.parse_decl(code)
print('Parsed AST', '----', e, '', sep='\n')

# Stage 1: Assign symbolic typenames
typing.assign_typenames(e.expr)
print('Typename assignment', '----',
      typing.show_type_assignment(e.expr), '', sep='\n')

# Stage 2: Generate a list of type equations
equations = []
typing.generate_equations(e.expr, equations)
print('Equations', '----', sep='\n')
for eq in equations:
    print('{:15} {:20} | {}'.format(str(eq.left), str(eq.right), eq.orig_node))

# Stage 3: Solve equations using unification
unifier = typing.unify_all_equations(equations)
print('', 'Inferred type', '----',
      typing.get_expression_type(e.expr, unifier, rename_types=True),
      sep='\n')

This will print out:

Code
----
foo f g x = if f(x == 1) then g(x) else 20

Parsed AST
----
Decl(foo, Lambda([f, g, x], If(App(f, [(x == 1)]), App(g, [x]), 20)))

Typename assignment
----
Lambda([f, g, x], If(App(f, [(x == 1)]), App(g, [x]), 20))   t0
If(App(f, [(x == 1)]), App(g, [x]), 20)                      t4
App(f, [(x == 1)])                                           t5
f                                                            t1
(x == 1)                                                     t6
x                                                            t3
1                                                            Int
App(g, [x])                                                  t7
g                                                            t2
x                                                            t3
20                                                           Int

Equations
----
Int             Int                  | 1
t3              Int                  | (x == 1)
Int             Int                  | (x == 1)
t6              Bool                 | (x == 1)
t1              (t6 -> t5)           | App(f, [(x == 1)])
t2              (t3 -> t7)           | App(g, [x])
Int             Int                  | 20
t5              Bool                 | If(App(f, [(x == 1)]), App(g, [x]), 20)
t4              t7                   | If(App(f, [(x == 1)]), App(g, [x]), 20)
t4              Int                  | If(App(f, [(x == 1)]), App(g, [x]), 20)
t0              ((t1, t2, t3) -> t4) | Lambda([f, g, x], If(App(f, [(x == 1)]), App(g, [x]), 20))

Inferred type
----
(((Bool -> Bool), (Int -> Int), Int) -> Int)

There are many more examples of type-inferred microml code snippets in the test file test_typing.py. Here's another example which is interesting:

> foo f x = if x then lambda t -> f(t) else lambda j -> f(x)
((Bool -> a), Bool) -> (Bool -> a)

The actual inference is implemented in typing.py, which is fairly well commented and should be easy to understand after reading this post. The trickiest part is probably the unification algorithm, but that one is just a slight adaptation of the algorithm presented in the previous post.

[1]

After this post was published, it was pointed out that another type checking / inference technique is already called bi-directional (see this paper for example); while it's related to Hindley-Milner (HM), it's a distinct method. Therefore, my terminology here can create a confusion.

I'll emphasize that my only use of the term "bi-directional" is to distinguish what HM does from the simpler "uni-directional" inference described at the beginning.