File-driven testing in Go

This is a quick post about a testing pattern I've found useful in multiple programming languages; Go's testing package makes it particularly easy to use and implement.

Table-driven tests are a well-known and recommended technique for structuring unit tests in Go. I assume the reader is familiar with table-driven tests; if not, check out this basic example or google for the many tutorials available online.

In some cases, the input to a test can be more data than we're willing to paste into a _test.go file, or the input can be non-text. For these cases, file-driven tests are a useful extension to the table-driven technique.

Suppose - for the sake of demonstration - that we want to test Go's go/format package. This package provides programmatic access to gofmt's capabilities with the Source function that takes unformatted Go text as input and produces formatted text. We could use table-driven tests for this, but some inputs may be large - whole Go files, or significant chunks thereof. It can be more convenient to have these inputs in external files the test can read and invoke format.Source on.

This toy GitHub repository shows how it's done. Here is the directory structure:

$ tree
.
├── go.mod
└── somepackage
    ├── somepackage_test.go
    └── testdata
        ├── funcs.golden
        ├── funcs.input
        ├── simple-expr.golden
        └── simple-expr.input

Our test code is in somepackage_test.go - we'll get to it shortly. Alongside it is the testdata directory with pairs of files named <name>.input and <name>.golden. Each pair serves as a test: the input file is fed into the formatter, and the formatter's output is compared to the golden file.

This is the entirety of our test code:

func TestFormatFiles(t *testing.T) {
  // Find the paths of all input files in the data directory.
  paths, err := filepath.Glob(filepath.Join("testdata", "*.input"))
  if err != nil {
    t.Fatal(err)
  }

  for _, path := range paths {
    _, filename := filepath.Split(path)
    testname := filename[:len(filename)-len(filepath.Ext(path))]

    // Each path turns into a test: the test name is the filename without the
    // extension.
    t.Run(testname, func(t *testing.T) {
      source, err := os.ReadFile(path)
      if err != nil {
        t.Fatal("error reading source file:", err)
      }

      // >>> This is the actual code under test.
      output, err := format.Source(source)
      if err != nil {
        t.Fatal("error formatting:", err)
      }
      // <<<

      // Each input file is expected to have a "golden output" file, with the
      // same path except the .input extension is replaced by .golden
      goldenfile := filepath.Join("testdata", testname+".golden")
      want, err := os.ReadFile(goldenfile)
      if err != nil {
        t.Fatal("error reading golden file:", err)
      }

      if !bytes.Equal(output, want) {
        t.Errorf("\n==== got:\n%s\n==== want:\n%s\n", output, want)
      }
    })
  }
}

The code is well commented, but here are a few highlights:

• The test file pairs are auto-discovered using filepath.Glob. Placing additional file pairs in the testdata directory will automatically ensure they are used by subsequent test executions. When we run go test, the current working directory will be set to the package that's being tested, so finding the testdata directory is easy.
• The testdata name is special for the Go toolchain, which will ignore files in it (so you can place files named *.go there, for example, and they won't be built or analyzed).
• For each file pair we create a subtest with T.Run. This means it's a separate test as far as the test runner is concerned - reported on its own in verbose output, can be run in parallel with other tests, etc.

If we run the tests, we get:

$ go test -v ./...
=== RUN   TestFormatFiles
=== RUN   TestFormatFiles/funcs
=== RUN   TestFormatFiles/simple-expr
--- PASS: TestFormatFiles (0.00s)
    --- PASS: TestFormatFiles/funcs (0.00s)
    --- PASS: TestFormatFiles/simple-expr (0.00s)
PASS
ok    example.com/somepackage 0.002s

Note how each input file in testdata generated its own test with a distinct name.

The "golden file" approach shown here is just one of the possible patterns for using file-driven tests. Often there is no separate file for the expected output, and instead the input file itself contains some special markers that drive the test's expectations. It's really dependent on the specific testing scenario; the Go project and subprojects (such as x/tools) use several variations of this testing pattern.