Locally patching dependencies in Go

Locally patching a dependency

To use a concrete example, suppose our module depends on the popular package go-cmp, that lets us deep-compare arbitrary Go values. Say we're debugging an intricate scenario and want to either:

• Add a log statement inside the dependency to see what our code is passing to it (e.g. "do I ever invoke cmp.Equal with these specific options?")
• Test a suspicion of a bug in the dependency by temporarily modifying its code and seeing if this has an effect on our module.

The Go module system makes this easy to accomplish; this post will demonstrate several way of doing this.

Setting up

Let's set up a test module to demonstrate this. The full code can be found on GitHub, or just follow along:

In a directory, run go mod init example.com (the module name is just a placeholder - it's a local experiment, we don't intend it to be imported or even published online). This creates a go.mod file; now, let's write this code:

package main

import (
  "fmt"

  "github.com/google/go-cmp/cmp"
  "github.com/google/go-cmp/cmp/cmpopts"
)

func main() {
  s1 := []int{42, 12, 23, 2}
  s2 := []int{12, 2, 23, 42}

  if cmp.Equal(s1, s2, cmpopts.SortSlices(intLess)) {
    fmt.Println("slices are equal")
  }
}

func intLess(x, y int) bool {
  return x < y
}

And then run go mod tidy; this should get the github.com/google/go-cmp dependency, and the go.mod file will look something like:

module example.com

go 1.22.2

require github.com/google/go-cmp v0.6.0

(your Go version and the dependency version will likely be different, of course)

Now, we'll download the dependency locally and patch it. Clone the https://github.com/google/go-cmp/ repository into a local directory; we'll call it $DEP (on my machine DEP=/home/eliben/test/go-cmp). Next, edit $DEP/cmp/compare.go to add a log statement:

func Equal(x, y interface{}, opts ...Option) bool {
  log.Println("options:", opts)
  s := newState(opts)
  s.compareAny(rootStep(x, y))
  return s.result.Equal()
}

If we run our test module now we don't see any effect yet:

$ go run .
slices are equal

This is to be expected! Go has no idea we've cloned the dependency locally and want it to be used in the build process of our test module. This is the next step.

Using a module replace directive

The most basic way to accomplish what we need is using a replace directive in the go.mod file of our test module.

In our module directory, run:

$ go mod edit -replace github.com/google/go-cmp=$DEP

If you look in your go.mod file, you'll see a new replace directive added there, redirecting uses of github.com/google/go-cmp to whatever directory DEP stands for on your machine.

If we now run the test module, it will pick up the patched dependency:

$ go run .
2024/06/29 06:57:17 options: [FilterValues(cmpopts.sliceSorter.filter, Transformer(cmpopts.SortSlices, cmpopts.sliceSorter.sort))]
slices are equal

Using Go workspaces

Go workspaces (go.work files) have been with us since version 1.18; a workspace makes it easier to work with multi-module repositories and large monorepos. It can also be leveraged to implement our use case very easily.

Get back to a clean go.mod file without a replace directive (you can either undo the change using source control, run go mod edit -dropreplace ... or just remove the replace directive from the go.mod file).

Now, run these commands in the test module's directory:

$ go work init
$ go work use . $DEP

This asks the Go tool to:

1. Initialize an empty workspace in the current directory; a go.work file will be created.
2. Add use directives to go.work for including the current directory . and the place where we checked out a local version of the dependency ($DEP).

If you look around, a new file was created - go.work; go.mod itself was not modified. If we run the module with go run ., we'll see that the local patch was picked up!

I like this approach a bit more than planting replace directives in the go.mod file, since it provides a cleaner separation between temporary patching and the module's actual source code. While go.mod files are checked into source control and provide a critical source of truth for building the module, go.work files aren't typically checked in and are used to set up a convenient local development environment. Using go.work for temporary patching is thus safer - it's more difficult to leave behind a replace directive in the go.mod file and commit it (this can cause all kinds of inconveniences when testing, for example).

Using gohack

gohack is a tool designed especially to address our use case; it predates Go workspaces. Start by installing it:

$ go install github.com/rogpeppe/gohack@latest

Now run:

$ gohack get github.com/google/go-cmp
github.com/google/go-cmp => $HOME/gohack/github.com/google/go-cmp

This invocation does two things:

1. Fetch the dependency's code and store it somewhere locally. You can control where these are stored by setting the $GOHACK env var; the default is $HOME/gohack.
2. Add a replace line to our go.mod file to point there.

Since gohack placed the dependency in a new location, we'll have to edit its cmp/compare.go file again to add the log statement. If we go run . in our test module, we'll see the change picked up.

It's also fairly easy to undo changes with the gohack undo command.

Which approach to use?

gohack can be useful in some cases where a quick check is all you need. Since gohack obtains the dependency on its own, it makes it a bit faster to use than cloning manually. That said, I'd be concerned about committing the replace line accidentally, which is why I think the workspace approach is safer (and also more explicit).

Update 2024-07-05: Sean Liao reminded me that go mod vendor is yet another way to accomplish this. This approach comes with its own tradeoffs; read the documentation to learn more.