Unix domain sockets in Go



When it comes to inter-process communication (IPC) between processes on the same Linux host, there are multiple options: FIFOs, pipes, shared memory, sockets and so on. One of the most interesting options is Unix Domain Sockets that combine the convenient API of sockets with the higher performance of the other single-host methods.

This post demonstrates some basic examples of using Unix domain sockets with Go and explores some benchmarks comparing them to TCP loop-back sockets.

Unix domain sockets (UDS)

Unix domain sockets (UDS) have a long history, going back to the original BSD socket specification in the 1980s. The Wikipedia definition is:

A Unix domain socket or IPC socket (inter-process communication socket) is a data communications endpoint for exchanging data between processes executing on the same host operating system.

UDS support streams (TCP equivalent) and datagrams (UDP equivalent); this post focuses on the stream APIs.

IPC with UDS looks very similar to IPC with regular TCP sockets using the loop-back interface (localhost or 127.0.0.1), but there is a key difference: performance. While the TCP loop-back interface can skip some of the complexities of the full TCP/IP network stack, it retains many others (ACKs, TCP flow control, and so on). These complexities are designed for reliable cross-machine communication, but on a single host they're an unnecessary burden. This post will explore some of the performance advantages of UDS.

There are some additional differences. For example, since UDS use paths in the filesystem as their addresses, we can use directory and file permissions to control access to sockets, simplifying authentication. I won't list all the differences here; for more information feel free to check out the Wikipedia link and additional resources like Beej's UNIX IPC guide.

The big disadvantage of UDS compared to TCP sockets is the single-host restriction, of course. For code written to use TCP sockets we only have to change the address from local to remote and everything keeps working. That said, the performance advantages of UDS are significant enough, and the API is similar enough to TCP sockets that it's quite possible to write code that supports both (UDS on a single host, TCP for remote IPC) with very little difficulty.

Using Unix domain sockets in Go

Let's start with a basic example of a server in Go that listens on a UNIX domain socket:

const SockAddr = "/tmp/echo.sock"

func echoServer(c net.Conn) {
    log.Printf("Client connected [%s]", c.RemoteAddr().Network())
    io.Copy(c, c)
    c.Close()
}

func main() {
    if err := os.RemoveAll(SockAddr); err != nil {
        log.Fatal(err)
    }

    l, err := net.Listen("unix", SockAddr)
    if err != nil {
        log.Fatal("listen error:", err)
    }
    defer l.Close()

    for {
        // Accept new connections, dispatching them to echoServer
        // in a goroutine.
        conn, err := l.Accept()
        if err != nil {
            log.Fatal("accept error:", err)
        }

        go echoServer(conn)
    }
}

UDS are identified with paths in the file system; for our server here we use /tmp/echo.sock. The server begins by removing this file if it exists, what is that about?

When servers shut down, the file representing the socket can remain in the file system unless the server did orderly cleanup after itself. If we re-run another server with the same socket path, we may get the error:

$ go run simple-echo-server.go
2019/02/08 05:41:33 listen error:listen unix /tmp/echo.sock: bind: address already in use

To prevent that, the server begins by removing the socket file, if it exists [1].

Now that the server is running, we can interact with it using Netcat, which can be asked to connect to UDS with the -U flag:

$ nc -U /tmp/echo.sock

Whatever you type in, the server will echo back. Press ^D to terminate the session. Alternatively, we can write a simple client in Go that connects to the server, sends it a message, waits for a response and exits. The full code for the client is here, but the important part is the connection:

c, err := net.Dial("unix", "/tmp/echo.sock")

We can see that writing UDS servers and clients is very similar to writing regular socket servers and clients. The only difference is having to pass "unix" as the network parameter of net.Listen and net.Dial; the rest of the code remains the same. Obviously, this makes it very easy to write generic server and client code that's independent of the actual kind of socket it's using.

HTTP and RPC protocols over UDS

Network protocols compose by design. High-level protocols, such as HTTP and various forms of RPC, don't particularly care about how the lower levels of the stack are implemented as long as certain guarantees are maintained.

Go's standard library comes with a small and useful rpc package that makes it trivial to throw together quick RPC servers and clients. Here's a simple server that has a single procedure defined:

const SockAddr = "/tmp/rpc.sock"

type Greeter struct {
}

func (g Greeter) Greet(name *string, reply *string) error {
    *reply = "Hello, " + *name
    return nil
}

func main() {
    if err := os.RemoveAll(SockAddr); err != nil {
        log.Fatal(err)
    }

    greeter := new(Greeter)
    rpc.Register(greeter)
    rpc.HandleHTTP()
    l, e := net.Listen("unix", SockAddr)
    if e != nil {
        log.Fatal("listen error:", e)
    }
    fmt.Println("Serving...")
    http.Serve(l, nil)
}

Note that we use the HTTP version of the server. It registers a HTTP handler with the http package, and the actual serving is done with the standard http.Serve. The network stack here looks something like this:

RPC / HTTP / Unix domain socket stack

An RPC client that can connect to the server shown above is available here. It uses the standard rpc.Client.Call method to connect to the server.

Benchmarking UDS compared to loop-back TCP sockets

Note: benchmarking is hard, so please take these results with a grain of salt. There's some more information on benchmarking different socket types on the Redis benchmarks page and in this paper, as well as many other resources online. I also found this set of benchmarks (written in C) instructive.

I'm running two kinds of benchmarks: one for latency, and one for throughput.

For latency, the full code of the benchmark is here. Run it with -help to see what the flags are, and the code should be very straightforward to grok. The idea is to ping-pong a small packet of data (128 bytes by default) between a server and a client. The client measures how long it takes to send one such message and receive one back, and takes that combined time as "twice the latency", averaging it over many messages.

On my machine, I see average latency of ~3.6 microseconds for TCP loop-back sockets, and ~2.3 microseconds for UDS.

The throughput/bandwidth benchmark is conceptually simpler than the latency benchmark. The server listens on a socket and grabs all the data it can get (and discards it). The client sends large packets (hundreds of KB or more) and measures how long each packet takes to send; the send is done synchronously and the client expects the whole message to be sent in a single call, so it's a good approximation of bandwidth if the packet size is large enough.

Obviously, the throughput measurement is more representative with larger messages. I tried increasing them until the throughput improvements tapered off.

For smaller packet sizes, I see UDS winning over TCP: 10 GB/sec compared to 9.4 GB/sec for 512K. For much larger packet sizes 16-32 MB, the difference becomes negligible (both taper off at about 13 GB/sec). Interestingly, for some packet sizes (like 64K), TCP sockets are winning on my machine.

For very small message sizes we're getting back to latency-dominated performance, so UDS is considerably faster (more than 2x the number of packets per second compared to TCP). In most cases I'd say that the latency measurements are more important - they're more applicable to things like RPC servers and databases. In some cases like streaming video or other "big data" over sockets, you may want to pick the packet sizes carefully to optimize the performance for the specific machine you're using.

This discussion has some really insightful information about why we should expect UDS to be faster. However, beware - it's from 2005 and much in Linux has changed since then.

Unix domain sockets in the real-world Go projects

I was curious to see if UDS are actually used in real-world Go projects. They sure are! A few minutes of browsing/searching Github quickly uncovered UDS servers in many components of the new Go-dominated cloud infrastructure: runc, moby (Docker), k8s, lstio - pretty much every project I looked at.

That makes sense - as the benchmarks demonstrate, there are significant performance advantages to using a UDS when the client and server are both on the same host. And the API of UDS and TCP sockets is so similar that the cost of supporting both interchangeably is quite small.


[1]For internet-domain socket, the same issue exists with ports that are marked taken by processes that die without cleanup. The SO_REUSEADDR socket option exists to address this problem.

Go JSON Cookbook



Recently I've gotten into answering Go questions on StackOverflow, and one of the patterns I noticed are many repetitive questions about JSON processing. The goal of this post is to collect a "cookbook" of JSON processing code and examples; think of it as a vastly expanded version of the JSON gobyexample page. It's a living document - I will update it once in a while when I find new patterns/problems people ask about.

The code samples here should be reasonably self-contained; if you want actual code, full go run-able files are available here.

Some background on JSON

The acronym JSON stands for JavaScript Object Notation. It originally came into existence as a serialization format for JS, and can actually be considered a subset of JS. That said, it's no longer considered good style to pass JSON to JS's eval(); in fact, some valid JSON is not valid JS. Newer editions of the ECMAScript standard provide JSON.stringify and JSON.parse for serialization and deserialization, respectively.

These days JSON is a very popular language-independent serialization format, with simple syntax that's described here. In brief, JSON has values that can be either strings, numbers, null, true/false boolean constants, arrays or objects. Arrays are linear, ordered collections of values; in Go they are mapped to slices. Objects are unordered sets of key/value pairs; in Go they mapped to maps. JSON object keys are strings, and values are arbitrary JSON values. Note that this is a recursive definition - objects can hold other objects, or lists, which themselves can hold other objects, etc. This should be very familiar to programmers coming from dynamic languages like Python, Perl or JavaScript, or from the Lisp family where such nested data structures are common and idiomatic.

Marshaling of basic Go data types

Go uses the term marshaling to refer to the kind of serialization done by converting Go data structures to JSON [1]. Therefore, the json package's main convenience functions are called json.Marshal and json.Unmarshal. These functions are generic, in the sense that they work with interface{} values, and they have the proper runtime logic to figure out which actual type is being (un)marshaled. Here is an example that uses the basic types:

boolS, _ := json.Marshal(true)
fmt.Println(string(boolS))
var b bool
if err := json.Unmarshal(boolS, &b); err != nil {
  panic(err)
}
fmt.Println("unmarshaled bool:", b)

intS, _ := json.Marshal(42)
fmt.Println(string(intS))
var i int
if err := json.Unmarshal(intS, &i); err != nil {
  panic(err)
}
fmt.Println("unmarshaled int:", i)

floatS, _ := json.Marshal(3.14159)
fmt.Println(string(floatS))
var f float64
if err := json.Unmarshal(floatS, &f); err != nil {
  panic(err)
}
fmt.Println("unmarshaled float64:", f)

stringS, _ := json.Marshal("golang")
fmt.Println(string(stringS))
var s string
if err := json.Unmarshal(stringS, &s); err != nil {
  panic(err)
}
fmt.Println("unmarshaled string:", s)

This will print:

true
unmarshaled bool: true
42
unmarshaled int: 42
3.14159
unmarshaled float64: 3.14159
"golang"
unmarshaled string: golang

Note that while JSON doesn't distinguish between integers and floating-point numbers, Go lets us do this by having the static type information on the objects passed into json.Marshal or json.Unmarshal. For more details on numbers see the "JSON numbers - ints or floats?" section.

Null and nil

Another basic type supported by JSON is null, which is just a way of saying "nothing to see here". In Go, nil pointers are marshaled to null:

nilS, _ := json.Marshal(nil)
fmt.Println(string(nilS))

Prints null. Demonstrating unmarshaling of null is a little bit trickier, because it's not clear what to pass to json.Unmarshal. Passing it any uninitialized pointer (nil) results in an error. A trick that works is:

var p interface{}
if err := json.Unmarshal(nilS, &p); err != nil {
  panic(err)
}
fmt.Println("unmarshaled null:", p)

Prints:

unmarshaled null: <nil>

(Un)marshaling with generic interface{} values

As was briefly mentioned above, the definitions of json.Marshal and json.Unmarshal are generic, in the sense that they use an interface{} for the Go object which is to be encoded or decoded into:

func Marshal(v interface{}) ([]byte, error)
func Unmarshal(data []byte, v interface{}) error

The samples above showcased how special logic inside these functions handles basic types that are known at compile time. We can do similar things using interface{} directly. While not overly useful for these basic types, this knowledge will come handy when we're discussing collection types (JSON representations of slices and maps) later on.

We saw earlier that to encode a boolean into JSON we can do:

boolS, _ := json.Marshal(true)

Alternatively, we can do:

var ii interface{}
ii = true
boolS, _ := json.Marshal(ii)
fmt.Println(string(boolS))

As far as json.Marshal is concerned, these two code samples are equivalent because the function really accepts a interface{}. It becomes a bit more interesting when unmarshaling. Where we previously did:

var b bool
if err := json.Unmarshal(boolS, &b); err != nil {
  panic(err)
}
fmt.Println("unmarshaled bool:", b)

We can also do:

var ib interface{}
if err := json.Unmarshal(boolS, &ib); err != nil {
  panic(err)
}

But now what do we do with ib? It's not a bool, so we can't treat it as such. Since we already know it should contain an bool, we can do a type assertion:

b := ib.(bool)
fmt.Println("unmarshaled bool:", b)

If we don't know what type to expect here exactly, we'll likely need a type switch:

switch v := ib.(type) {
case bool:
  fmt.Println("it's a bool:", v)
case float64:
  fmt.Println("it's a float:", v)
// other possible types enumerated...
default:
  panic("can't figure out the type")
}

JSON numbers - ints or floats?

One common issue with decoding JSON is distinguishing between different numeric types. Unlike Go, which has separate types for integers and floats, JavaScript only has floats; this fact is reflected in JSON. The JSON standard doesn't acknowledge the distinction between the two, and treats both as "numbers", though it's clear from the specification that the more general type (floating point) is inferred.

As long as we know the type of field to decode statically, everything will work fine. As the very first example in this post demonstrates, when we pass a pointer to int to Unmarshal, it will know to parse properly into an integer. But what happens when we don't know the type at compile time?

When using generic interface{} decoding, floating point is always chosen. Consider this:

func main() {
  s := []byte("1234")
  var inum interface{}
  if err := json.Unmarshal(s, &inum); err != nil {
    panic(err)
  }
  switch v := inum.(type) {
  case int:
    fmt.Println("it's an int:", v)
  case float64:
    fmt.Println("it's a float:", v)
  // other possible types enumerated...
  default:
    panic("can't figure out the type")
  }
}

It will print:

it's a float: 1234

This is not a bug; it's the logical thing for Unmarshal to do, given that it doesn't know what type to expect. 1234 looks like an integer, but it might as well be a float with the decimal point omitted. Unmarshal has to decode the most general type based on the JSON specification.

If this is a real issue, one way to work around it is to use the alternative json.Decoder API for unmarshaling. This API is slightly different from json.Unmarhsal; it's designed to parse JSON streams, which could result from reading over HTTP, for example. Here's the same code using Decoder:

d := json.NewDecoder(bytes.NewReader(s))
var ii interface{}
if err := d.Decode(&ii); err != nil {
  panic(err)
}
switch v := ii.(type) {
case int:
  fmt.Println("it's an int:", v)
case float64:
  fmt.Println("it's a float:", v)
// other possible types enumerated...
default:
  panic("can't figure out the type")
}

It gives a similar result to using Unmarshal. However, here's the twist. Decoder has an option to not parse numbers into concrete types. Instead, a number be left unparsed as a json.Number, which is a just a string used to represent number literals. This is accomplished by calling Decoder.UseNumber(), as follows:

d := json.NewDecoder(bytes.NewReader(s))
var ii interface{}
d.UseNumber()                              // <-- UseNumber
if err := d.Decode(&ii); err != nil {
  panic(err)
}
switch v := ii.(type) {
case int:
  fmt.Println("it's an int:", v)
case float64:
  fmt.Println("it's a float:", v)
case json.Number:
  fmt.Println("it's a string:", v)
// other possible types enumerated...
default:
  panic("can't figure out the type")
}

Now this will print:

it's a string: 1234

And we're free to parse the string as we wish, for example with strconv.Atoi.

You may think this is unnecessary - can't we just convert the float64 read by Unmarshal into an int? Things are not so simple, however. Floating point numbers have limited representation accuracy, and for big integers we may get wrong results. We might even want to marshal arbitrary precision integers (big.Int), and these also have to be properly parsed to not lose precision.

JSON arrays and Go slices

Go slices are encoded to JSON arrays, and vice-versa:

sS, _ := json.Marshal([]string{"broccoli", "almonds", "banana"})
fmt.Println(string(sS))

var s []string
if err := json.Unmarshal(sS, &s); err != nil {
  panic(err)
}
fmt.Println("unmarshaled []string:", s)

Prints:

["broccoli","almonds","banana"]
unmarshaled []string: [broccoli almonds banana]

When we unmarshal the JSON bytes above, we used a []string since we knew all the elements of the JSON array are strings. But what happens when JSON array elements have different types? While in Go the types of all slice elements has to be the uniform, the same is not true in JSON (due to its JavaScript roots). Let's try this:

variedEncodedSlice := []byte(`["broccoli", true]`)
var s2 []string
if err := json.Unmarshal(variedEncodedSlice, &s2); err != nil {
  panic(err)
}
fmt.Println("unmarshaled []string:", s2)

This code panics:

panic: json: cannot unmarshal bool into Go value of type string

The error message makes sense: we gave json.Unmarshal a []string to unmarshal into, but the JSON bytes contain a bool. We can't unmarshal into an []bool, for a similar reason. So what is there to do?

This is where generic JSON comes in again. If we don't know - ahead of time - the types of elements contained in a JSON array we're unmarshaling, we have to fall back to generic interface{}s and type switches:

var iis []interface{}
if err := json.Unmarshal(variedEncodedSlice, &iis); err != nil {
  panic(err)
}
fmt.Println("unmarshalled slice of length:", len(iis))
for i, e := range iis {
  fmt.Printf("decoding element %d\n", i)
  switch v := e.(type) {
  case bool:
    fmt.Println("  it's a bool:", v)
  case string:
    fmt.Println("  it's a string:", v)
  // other possible types enumerated...
  default:
    panic("can't figure out the type")
  }
}

This outputs:

unmarshalled slice of length: 2
decoding element 0
  it's a string: broccoli
decoding element 1
  it's a bool: true

JSON objects and Go maps

Go maps are encoded to JSON objects, and vice-versa:

mS, _ := json.Marshal(map[string]bool{"almonds": false, "cashews": true})
fmt.Println(string(mS))

var m map[string]bool
if err := json.Unmarshal(mS, &m); err != nil {
  panic(err)
}
fmt.Println("unmarshaled map[string]bool:", m)

Prints:

{"almonds":false,"cashews":true}
unmarshaled map[string]bool: map[almonds:false cashews:true]

Similarly to the case of slices, this works well as long as the types of JSON elements are known ahead of time. If these types are not known, or values can have one of several types, we'll need to use generic capabilities, by unmarshaling into an interface{} and following up with type assertions or switches.

We've seen how to do these in the slice example, so let's play with something slightly different now. JSON objects are sometimes nested, and we don't even know at compile-time what their level of nesting is. Consider this sample JSON:

{
  "foo": true,
  "bar": false,
  "baz": {
    "next": true,
    "prev": {
      "fizz": true,
      "buzz": false
    },
    "top": false
  }
}

Say we want to find the key "fizz" in it, and to see what it maps to. How can we do that?

Let's think this through. First, it's obvious that we'll have to use interface{} unmarshaling, because the values of keys in each object can have different types (some are booleans, some are nested objects). Second, since JSON is a tree structure, a recursive approach is natural. Here's a function that will do that:

// findNested looks for a key named s in map m. If values in m map to other
// maps, findNested looks into them recursively. Returns true if found, and
// the value found.
func findNested(m map[string]interface{}, s string) (bool, interface{}) {
  // Try to find key s at this level
  for k, v := range m {
    if k == s {
      return true, v
    }
  }
  // Not found on this level, so try to find it nested
  for _, v := range m {
    nm, ok := v.(map[string]interface{})
    if ok {
      found, val := findNested(nm, s)
      if found {
        return found, val
      }
    }
  }
  // Not found recursively
  return false, nil
}

And here's how we can use it:

bb := []byte(`
{
  "foo": true,
  "bar": false,
  "baz": {
    "next": true,
    "prev": {
      "fizz": true,
      "buzz": false
    },
    "top": false
  }
}`)

var ii interface{}
if err := json.Unmarshal(bb, &ii); err != nil {
  panic(err)
}
mi := ii.(map[string]interface{})
ok, fizzVal := findNested(mi, "fizz")
if ok {
  fmt.Println("found fizz! value is", fizzVal)
}

Golang structs as JSON objects

Nested slices and maps are great, but in Go it's idiomatic to assign more semantics to structured data with structs. Go's json package supports marshaling structs into JSON objects and vice versa. Here's a simple example:

type Food struct {
  Id             int
  Name           string
  FatPerServ     float64
  ProteinPerServ float64
  CarbPerServ    float64
}

func main() {
  f := Food{200403, "Broccoli", 0.3, 2.5, 3.5}
  fS, _ := json.MarshalIndent(f, "", "  ")
  fmt.Println(string(fS))

  var fD Food
  if err := json.Unmarshal(fS, &fD); err != nil {
    panic(err)
  }
  fmt.Println("unmarshaled Food:", fD)
}

It will print:

{
  "Id": 200403,
  "Name": "Broccoli",
  "FatPerServ": 0.3,
  "ProteinPerServ": 2.5,
  "CarbPerServ": 3.5
}
unmarshaled Food: {200403 Broccoli 0.3 2.5 3.5}

Here we're using the MarshalIndent method that indents JSON output for easier visual scanning.

You'll notice that the JSON objects have their key names set from the struct field names automatically. This is nice for low-effort dumping, but isn't always satisfactory in real applications. We often don't control the format of the JSON data we're consuming, so we may get something like:

{
  "id": 200403,
  "name": "Broccoli",
  "fat_per_serv": 0.3,
  "protein_per_serv": 2.5,
  "carb_per_serv": 3.5
}

If we call json.Unmarshal for data like this, it will expect a struct with similarly named fields; but these names aren't idiomatic in Go, and moreover they start with lowercase so they won't even be visible outside the struct's own methods. We have a problem - we either sacrifice the style of our Go code, or have to enforce schemas on JSON we don't necessarily control.

The solution is to use custom field tags, which is an esoteric feature of Go that was designed specifically for such use cases. Here's a complete example:

type Food struct {
  Id             int     `json:"id"`                // <--- Field tags
  Name           string  `json:"name"`
  FatPerServ     float64 `json:"fat_per_serv"`
  ProteinPerServ float64 `json:"protein_per_serv"`
  CarbPerServ    float64 `json:"carb_per_serv"`
}

func main() {
  f := Food{200403, "Broccoli", 0.3, 2.5, 3.5}
  fS, _ := json.MarshalIndent(f, "", "  ")
  fmt.Println(string(fS))

  var fD Food
  if err := json.Unmarshal(fS, &fD); err != nil {
    panic(err)
  }
  fmt.Println("unmarshaled Food:", fD)
}

It prints:

{
  "id": 200403,
  "name": "Broccoli",
  "fat_per_serv": 0.3,
  "protein_per_serv": 2.5,
  "carb_per_serv": 3.5
}
unmarshaled Food: {200403 Broccoli 0.3 2.5 3.5}

Field tags let us map between the Go internal view of the struct's fields and its external materialization as a JSON object.

These techniques work just as well with nested structs. Check out these code samples: sample 1, sample 2.

Partial encoding and decoding of structs

It's common for JSON data to omit some fields that are then assumed to not exist or take on their default values. Think about passing many options, where the full list of options is so large that a lot of time and bandwidth would be wasted to transfer them all fully; usually we only want to modify a small number of options for every given call.

The json package supports this with partial unmarshaling. Here's an example using the Food struct shown above:

func main() {
  bb := []byte(`
  {
    "Name": "Broccoli",
    "FatPerServ": 0.3,
    "ProteinPerServ": 2.5,
    "CarbPerServ": 3.5
  }`)
  var fD Food
  if err := json.Unmarshal(bb, &fD); err != nil {
    panic(err)
  }
  fmt.Println("unmarshaled Food:", fD)
}

Note that the JSON string doesn't have the Id field populated. The result will be:

unmarshaled Food: {0 Broccoli 0.3 2.5 3.5}

The unmarshaling is successful, and fD.Id is left at the default value for its type (0 for numbers, empty string for strings, etc). This behavior can be controlled via the Decoder.DisallowUnknownFields method when using the json.Decoder API.

For a similar effect during marshaling, we can use the special "omitempty" field tag; it tells the json package to not emit a struct field if it has the default value for its type. Here's an example:

type Food struct {
  Id             int     `json:"id,omitempty"`
  Name           string  `json:"name"`
  FatPerServ     float64 `json:"fat_per_serv"`
  ProteinPerServ float64 `json:"protein_per_serv"`
  CarbPerServ    float64 `json:"carb_per_serv"`
}

func main() {
  f := Food{0, "Broccoli", 0.3, 2.5, 3.5}
  fS, _ := json.MarshalIndent(f, "", "  ")
  fmt.Println(string(fS))
}
{
  "name": "Broccoli",
  "fat_per_serv": 0.3,
  "protein_per_serv": 2.5,
  "carb_per_serv": 3.5
}

Note how the id field is left out of the JSON, because it was given the empty value 0.

We can also tell the encoder to omit certain fields. For example, we may have a struct with a field that should be kept private to the application and not sent over the wire. Even when the field has a non-empty value, we want it out of the serialized JSON string. We can do this with the json:"-" field tag:

type Account struct {
    Name     string
    Password string `json:"-"`
    Balance  float64
}

func main() {
    joe := Account{Name: "Joe", Password: "123456", Balance: 102.4}
    s, _ := json.Marshal(joe)
    fmt.Println(string(s))
}

Delayed parsing with json.RawMessage

Sometimes the data you need to parse is not on the top level of the JSON string, and/or you'd like to ignore a lot of the JSON contents, focusing just on the piece you need to parse. Consider this JSON string:

{
        "event": {"name": "joe", "url": "event://101"},
        "otherstuff": 15.2,
        "anotherstuff": 100
}

And suppose we're interested only in the event key, as we already have a structure to parse it into:

type Event struct {
    Name string `json:"name"`
    Url  string `json:"url"`
}

How do we do that? The json module relies on static typing quite a bit, unless we go full generic with interface{}. But in that case, we may need to convert large maps into large structs manually, which is undesirable.

The solution is json.RawMessage, which exists for this purpose. It tells the json module to not parse some parts of the string and leave them as strings, which we can then parse again later. Here's a complete solution to the issue discussed above:

type Event struct {
    Name string `json:"name"`
    Url  string `json:"url"`
}

func main() {
    bb := []byte(`
    {
        "event": {"name": "joe", "url": "event://101"},
        "otherstuff": 15.2,
        "anotherstuff": 100
    }`)

    var m map[string]json.RawMessage
    if err := json.Unmarshal(bb, &m); err != nil {
        panic(err)
    }

    if eventRaw, ok := m["event"]; ok {
        var event Event
        if err := json.Unmarshal(eventRaw, &event); err != nil {
            panic(err)
        }
        fmt.Println("Parsed Event:", event)
    } else {
        fmt.Println("Can't find 'event' key in JSON")
    }
}

JSON and pointer/reference types

The json package has special handling for pointer and reference types. Consider this sample structure:

type NamePtr struct {
  Id   int
  Name *string
}

We can marshal it as follows:

name := "Sam"
np := NamePtr{101, &name}

npS, _ := json.Marshal(np)
fmt.Println(string(npS))

This will print:

{"Id":101,"Name":"Sam"}

This works because json.Marshal does the right thing here - it "sees through" the pointer to string and emits the string itself as the "Name" field. It works in reverse as well:

var npD NamePtr
if err := json.Unmarshal(npS, &npD); err != nil {
  panic(err)
}
fmt.Println(npD.Id, *npD.Name)

Note that when we create npD, its Name field is initialized with the default value - a nil for pointers. json.Unmarshal allocates an actual value and sets the pointer to its address when unmarshaling. If Name is not present in the JSON string being decoded, the pointer will be left as nil.

The same applies to other reference types, like slices:

type BoolAndVals struct {
  Fresh bool
  Vals  []float64
}

func main() {
  bb := []byte(`
  {
    "Fresh": true,
    "Vals": [1.2, 3.24, 18.99]
  }`)
  var bvD BoolAndVals
  if err := json.Unmarshal(bb, &bvD); err != nil {
    panic(err)
  }
  fmt.Println(bvD)
}

This will print:

{true [1.2 3.24 18.99]}

When we declare the variable bvD, its Vals field is an unallocated slice, but json.Unmarshal will allocate it for us if the Vals field is present in the decoded JSON object.

This behavior is very useful for being able to multiplex several struct types in a single container, implementing a sum type. Here's a complete example:

type RequestBodyFoo struct {
  Name    string
  Balance float64
}

type RequestBodyBar struct {
  Id  int
  Ref int
}

type Request struct {
  Foo *RequestBodyFoo
  Bar *RequestBodyBar
}

func (r *Request) Show() {
  if r.Foo != nil {
    fmt.Println("Request has Foo:", *r.Foo)
  }
  if r.Bar != nil {
    fmt.Println("Request has Bar:", *r.Bar)
  }
}

func main() {
  bb := []byte(`
  {
    "Foo": {"Name": "joe", "balance": 4591.25}
  }
  `)

  var req1 Request
  if err := json.Unmarshal(bb, &req1); err != nil {
    panic(err)
  }
  req1.Show()

  bb = []byte(`
  {
    "Bar": {"Id": 128992, "Ref": 801472}
  }
  `)
  var req2 Request
  if err := json.Unmarshal(bb, &req2); err != nil {
    panic(err)
  }
  req2.Show()
}

This prints:

Request has Foo: {joe 4591.25}
Request has Bar: {128992 801472}

Parsing JSON streams with Decoder

We've briefly seen json.Decoder before because it has some extended functionality that json.Marshal lacks. Now let's see how to use it more idiomatically, for parsing JSON streams.

json.Marshal is a convenience function. It takes a full byte slice and parses its contents. But JSON data often arrives from some streaming medium like a socket, and it's occasionally useful to parse it in a more fine-grained manner. For example, the stream may contain a sequence of JSON values and we want to parse each value as soon as it arrives - json.Marshal would require us to consume the whole stream before parsing it.

Here's an example that accomplishes this. It showcases using the More and Token methods for fine-grained parsing of a stream:

func main() {
  const s = `
  [
    {"almonds": false},
    {"cashews": true},
    {"walnuts": false}
  ]`
  dec := json.NewDecoder(strings.NewReader(s))

  t, err := dec.Token()
  if err != nil {
    panic(err)
  }
  if t != json.Delim('[') {
    panic("Expected '[' delimiter")
  }

  for dec.More() {
    var m map[string]bool
    err := dec.Decode(&m)
    if err != nil {
      panic(err)
    }

    fmt.Println("decoded", m)
  }

  t, err = dec.Token()
  if err != nil {
    panic(err)
  }
  if t != json.Delim(']') {
    panic("Expected ']' delimiter")
  }
}

Since json.NewDecoder accepts any io.Reader, it's very versatile. Many Go types implement the io.Reader interface, so decoders can be hooked up to sockets, files or even something like a decompressed reading using the zip package.

Encoding JSON streams with Encoder

So far we've done marshaling with the json.Marshal function, which takes an object and produces a slice of bytes. The json package also offers a lower-level, more flexible API called Encoder, which can encode objects into a stream, without having to materialize a temporary []byte buffer. In Go, streams can be conveniently hooked together using interfaces provided by the io package, such as io.Writer. Here's a complete example that emits objects in JSON while also encoding them with base64. Follow the comments in the code:

package main

import (
  "encoding/base64"
  "encoding/json"
  "os"
)

func main() {
  // Create an io.Writer that encodes bytes written into it as base64 and emits
  // them to stdout.
  base64writer := base64.NewEncoder(base64.StdEncoding, os.Stdout)

  // Create a new JSON encoder, hooking up its output to base64writer.
  je := json.NewEncoder(base64writer)
  je.Encode(map[string]float64{"foo": 4.12, "pi": 3.14159})

  // Flush any partially encoded blocks left in the base64 encoder.
  base64writer.Close()
}

Custom JSON encoding with the Marshaler interface

Suppose you have a type which you want to encode in some special way, and the supported field tags are insufficient to customize it the way you need. For example, you may want your numbers to be encoded as hexadecimal (maybe because of some limitations of the API on the other end).

For full customization, your type can implement the json.Marshaler interface:

type Marshaler interface {
    MarshalJSON() ([]byte, error)
}

Here's an example:

type Account struct {
    Id   int32
    Name string
}

func (a Account) MarshalJSON() ([]byte, error) {
    m := map[string]string{
        "id":   fmt.Sprintf("0x%08x", a.Id),
        "name": a.Name,
    }
    return json.Marshal(m)
}

func main() {
    joe := Account{Id: 123, Name: "Joe"}
    fmt.Println(joe)

    s, _ := json.Marshal(joe)
    fmt.Println(string(s))
}

This prints:

{"id":"0x0000007b","name":"Joe"}

Similarly, we could customize decoding by implementing the json.Unmarshaler interface.


[1]Though Go is not overly consistent in this terminology, and encoding/decoding is also used for the same purpose. This post will also alternate between marshaling/encoding.

On concurrency in Go HTTP servers



Go's built-in net/http package is convenient, solid and performant, making it easy to write production-grade web servers. To be performant, net/http automatically employs concurrency; while this is great for high loads, it can also lead to some gotchas. In this post I want to explore this topic a bit.

Ensuring safe concurrent access from handlers to data

Let's start with a very simple example of a HTTP server that implements a table of counters accessible to the user. We can create counters (or set values of existing counters) with the set?name=N&val=V query, get their values with the get?name=N query and increment them with the inc?name=N query. Here's a simple interaction recorded with a server running in the background on port 8000:

$ curl "localhost:8000/set?name=x&val=0"
ok
$ curl "localhost:8000/get?name=x"
x: 0
$ curl "localhost:8000/inc?name=x"
ok
$ curl "localhost:8000/get?name=x"
x: 1

And a basic server implementing this functionality:

package main

import (
  "fmt"
  "log"
  "net/http"
  "os"
  "strconv"
)

type CounterStore struct {
  counters map[string]int
}

func (cs CounterStore) get(w http.ResponseWriter, req *http.Request) {
  log.Printf("get %v", req)
  name := req.URL.Query().Get("name")
  if val, ok := cs.counters[name]; ok {
    fmt.Fprintf(w, "%s: %d\n", name, val)
  } else {
    fmt.Fprintf(w, "%s not found\n", name)
  }
}

func (cs CounterStore) set(w http.ResponseWriter, req *http.Request) {
  log.Printf("set %v", req)
  name := req.URL.Query().Get("name")
  val := req.URL.Query().Get("val")
  intval, err := strconv.Atoi(val)
  if err != nil {
    fmt.Fprintf(w, "%s\n", err)
  } else {
    cs.counters[name] = intval
    fmt.Fprintf(w, "ok\n")
  }
}

func (cs CounterStore) inc(w http.ResponseWriter, req *http.Request) {
  log.Printf("inc %v", req)
  name := req.URL.Query().Get("name")
  if _, ok := cs.counters[name]; ok {
    cs.counters[name]++
    fmt.Fprintf(w, "ok\n")
  } else {
    fmt.Fprintf(w, "%s not found\n", name)
  }
}

func main() {
  store := CounterStore{counters: map[string]int{"i": 0, "j": 0}}
  http.HandleFunc("/get", store.get)
  http.HandleFunc("/set", store.set)
  http.HandleFunc("/inc", store.inc)

  portnum := 8000
  if len(os.Args) > 1 {
    portnum, _ = strconv.Atoi(os.Args[1])
  }
  log.Printf("Going to listen on port %d\n", portnum)
  log.Fatal(http.ListenAndServe("localhost:"+strconv.Itoa(portnum), nil))
}

This code is simple; too simple, in fact, to the degree that it's wrong. Our sample curl session sends all its requests in a serial manner. The problems appear when concurrent connections are present, however. A good way to simulate many concurrent connections is with ApacheBench:

$ ab -n 20000 -c 200 "127.0.0.1:8000/inc?name=i"

Benchmarking 127.0.0.1 (be patient)
Completed 2000 requests
Completed 4000 requests

Test aborted after 10 failures

apr_socket_connect(): Connection reset by peer (104)
Total of 4622 requests completed

Oops... what happened? Checking the logs of the Go server, we'll see something like:

<normal server logs>
fatal error: concurrent map writes

goroutine 6118 [running]:
runtime.throw(0x6b0a5c, 0x15)
  /usr/local/go/src/runtime/panic.go:608 +0x72 fp=0xc00060dba8 sp=0xc00060db78 pc=0x42ba12

Reviewing our code, the problem is apparent. The request handlers can run concurrently but they all manipulate a shared CounterStore. For example, the inc handler is being called concurrently for multiple requests and attempts to mutate the map. This leads to a race condition since in Go, map operations are not atomic. Luckily, the Go runtime detects this and dies with a helpful message; it would be worse if data were silently corrupted.

The simplest solution is to serialize all map accesses using a mutex. Here's an excerpt from a complete code sample that implements this:

type CounterStore struct {
  sync.Mutex
  counters map[string]int
}

func (cs *CounterStore) inc(w http.ResponseWriter, req *http.Request) {
  log.Printf("inc %v", req)
  cs.Lock()
  defer cs.Unlock()
  name := req.URL.Query().Get("name")
  if _, ok := cs.counters[name]; ok {
    cs.counters[name]++
    fmt.Fprintf(w, "ok\n")
  } else {
    fmt.Fprintf(w, "%s not found\n", name)
  }
}

Note two changes:

  1. We embed a sync.Mutex in CounterStore, and each handler starts by locking the mutex (and deferring an unlock).
  2. We change the receiver inc is defined on to a pointer *CounterStore. In fact, the previous version of the code was wrong in this respect - methods that modify data should always be defined with pointer receivers. We got lucky that the data was shared at all with value receivers because maps are reference types. Pointer receivers are particularly critical when mutexes are involved.

If we rerun the ab benchmark with this fixed server, it passes; the race condition is gone.

Synchronizing with channels vs. mutexes

To programmers experienced in most other languages, adding a mutex to synchronize accesses to a CounterStore is a natural solution. One of the mottos of Go is, however, "Share memory by communicating, don't communicate by sharing memory". Does this apply here?

Instead of mutexes, we could use channels to synchronize access to shared data. This code sample reimplements the mutex-using server to use channels instead. We start by defining a "counter manager" which is a background goroutine with access to a closure that stores the actual data:

type CommandType int

const (
  GetCommand = iota
  SetCommand
  IncCommand
)

type Command struct {
  ty        CommandType
  name      string
  val       int
  replyChan chan int
}

func startCounterManager(initvals map[string]int) chan<- Command {
  counters := make(map[string]int)
  for k, v := range initvals {
    counters[k] = v
  }

  cmds := make(chan Command)

  go func() {
    for cmd := range cmds {
      switch cmd.ty {
      case GetCommand:
        if val, ok := counters[cmd.name]; ok {
          cmd.replyChan <- val
        } else {
          cmd.replyChan <- -1
        }
      case SetCommand:
        counters[cmd.name] = cmd.val
        cmd.replyChan <- cmd.val
      case IncCommand:
        if _, ok := counters[cmd.name]; ok {
          counters[cmd.name]++
          cmd.replyChan <- counters[cmd.name]
        } else {
          cmd.replyChan <- -1
        }
      default:
        log.Fatal("unknown command type", cmd.ty)
      }
    }
  }()
  return cmds
}

Instead of accessing the map of counters directly, handlers will send Commands on a channel and will receive replies on a reply channel they provide.

The shared object for the handlers will now be a Server:

type Server struct {
  cmds chan<- Command
}

And here's the inc handler:

func (s *Server) inc(w http.ResponseWriter, req *http.Request) {
  log.Printf("inc %v", req)
  name := req.URL.Query().Get("name")
  replyChan := make(chan int)
  s.cmds <- Command{ty: IncCommand, name: name, replyChan: replyChan}

  reply := <-replyChan
  if reply >= 0 {
    fmt.Fprintf(w, "ok\n")
  } else {
    fmt.Fprintf(w, "%s not found\n", name)
  }
}

Each handler deals with the manager synchronously; the Command send is blocking, and so is the read from the reply channel. But note - not a mutex in sight! Mutual exclusion is accomplished by a single goroutine having access to the actual data.

While it certainly looks like an interesting technique, for our particular use case this approach seems like an overkill. In fact, overuse of channels is one of the common gotchas for Go beginners. Quoting from the Go Wiki entry:

Most locking issues can be solved using either channels or traditional locks.

So which should you use?

Use whichever is most expressive and/or most simple.

It then goes on to suggest that mutexes are preferable for protecting shared state. I tend to agree, since a mutex feels more natural here.

Limiting the degree of concurrency for our server

In addition to synchronization, another aspect of concurrency we have to worry about is overloading of the server. Imagine exposing a server to the internet - without any safeguards it would be fairly easy to bring it down with a denial of service (DoS) attack. That could happend even unintentionally, if we didn't provision the proper computing power behind the service. In these cases failure is unavoidable but should be graceful.

It's very easy to do these things in Go, and there are many strategies. One of the simplest is rate limiting, which can mean either restricting the number of simultaneous connections, or restricting the number of connections per unit of time. For the former, we can add some middleware (full code here):

// limitNumClients is HTTP handling middleware that ensures no more than
// maxClients requests are passed concurrently to the given handler f.
func limitNumClients(f http.HandlerFunc, maxClients int) http.HandlerFunc {
  // Counting semaphore using a buffered channel
  sema := make(chan struct{}, maxClients)

  return func(w http.ResponseWriter, req *http.Request) {
    sema <- struct{}{}
    defer func() { <-sema }()
    f(w, req)
  }
}

And wrap a handler using it:

// Limit to max 10 connections for this handler.
http.HandleFunc("/inc", limitNumClients(store.inc, 10))

This ensures that no more than 10 simultaneous clients will be allowed to invoke the inc handler. It should be simple to extend this approach to share a single limiting channel between multiple handlers. Or we could use a more heavy-handed solution and limit the number of simulatenous connections at the listener level, using something like netutil.LimitListener.

An alternative approach to rate limiting is time-based. Instead of saying "no more than 10 requests at a time", we can say "no more than one request in a 50 millisecond period". Go provides the convenient time.Tick channel that makes it easy to implement, as this Go example demonstrates. Adjusting this approach to our sample server is left as an exercise to the reader.

Appendix: where net.http goes concurrent

This is a brief technical appendix, for folks interested following through the code of Go's net/http package to see where the concurrency is actually implemented.

The relevant source code for the serving part of net/http is in https://golang.org/src/net/http/server.go

ListenAndServe invokes Serve, which calls Accept in a loop. Accept is similar to the socket accept syscall, creating a new connection whenever a new client is accepted; the connection is then used for interacting with the client. This connection is type conn in the server.go file, a private type with server state for each client connection. Its serve method actually serves a connection, pretty much as you would expect; it reads some data from the client and invokes the user-suppied handler depending on the path.

http.Server.Serve calls conn.serve as:

go c.serve(ctx)

Wherein lies the concurrency. From this point on, a separate goroutine handles this connection. This is why multiple goroutines could be executing user-specified handlers (including a single handler) at any given time.