go / test

Assertions come from is. Database tests insert fixtures through a test helper. CI reuses the Go caches across runs, with two flags that keep the result cache warm.

Assertions

func TestRename(t *testing.T) {
	is := is.New(t)

	co, err := Rename(id, "Acme")

	is.NoErr(err)
	is.Eq(co.Name, "Acme")
}

A failure reads:

rename_test.go:9: co.Name = Beta, want Acme

The variable shadows the package so a call site reads as English.

New(t) fails fatally on the first failure. NewRelaxed(t) records and continues, for a test that reports every mismatch in a loop.

Eq(x, nil) fails for a nil *T, because a typed nil boxed into any is a non-nil interface. Nil handles it.

True(got == want) compiles, prints false, want true, and loses both values.

Eq takes any

A method cannot have type parameters, so Eq takes any and gives up compile-time type checking:

is.Eq(gotInt64, 3) // builds, then fails: int64 3 against int 3

When the two values format the same, the failure prints %T too. The fix is a typed literal, int64(3).

Eq uses reflect.Value.Equal where both values are comparable, so pointers compare by identity. It uses reflect.DeepEqual elsewhere, so slices and maps compare by contents.

Label from source

co.Name is the source text of the assertion's first argument, read back from the test file. There is no message to write.

The values always print. A failed file read degrades to got X, want Y. A literal argument gets no label, because []int{1, 2} = []int{1, 2} is noise.

Under -trimpath, which the cache needs, runtime.Caller returns a module-relative path that does not open. filepath.Base of it does open, because go test runs each binary in its package directory.

The package parses the whole file and takes the call node that covers the line. So a call spread over several lines keeps its label. The node must be a call to the helper. This check rejects a file with the same name from another package.

gotest.tools/v3/assert also reads the source, but opens the path runtime.Caller returns, so under go test -trimpath its parse fails. matryer/is prints the trailing comment, which is still a hand-written message.

Fixtures

I use is beside database test fixtures:

func TestRename(t *testing.T) {
	t.Parallel()
	is := is.New(t)
	db := test.NewDB(t)
	co := db.InsertCompany()

	err := Rename(t.Context(), db.DB, co.ID, "Acme")

	is.NoErr(err)
	is.Eq(db.FindCompany(co.ID).Name, "Acme")
}

db wraps the database connection and holds fixture helpers such as db.InsertCompany(). is holds the assertions.

I keep the two helpers separate. db.Eq reads like a database query instead of a test assertion.

The test runs the function and checks the result with is. Each fixture method inserts a row into a test transaction:

type CompanyOpts struct {
	Name   string
	Status string
}

type CompanyRow struct {
	ID   int64  `db:"id"`
	Name string `db:"name"`
}

func (db *DB) InsertCompany(opts ...CompanyOpts) CompanyRow {
	db.t.Helper()
	var o CompanyOpts
	if len(opts) > 0 {
		o = opts[0]
	}
	if o.Name == "" {
		o.Name = "Test Co"
	}
	if o.Status == "" {
		o.Status = "active"
	}

	var row CompanyRow
	err := db.QueryRow(db.ctx, `
		INSERT INTO companies (name, status)
		VALUES ($1, $2)
		RETURNING id, name
	`, o.Name, o.Status).Scan(&row.ID, &row.Name)
	if err != nil {
		db.t.Fatalf("insert company: %v", err)
	}
	return row
}

The options struct sets defaults that satisfy database constraints. A test overrides only the fields it tests.

The method returns the inserted row with its generated ID. If the insert fails, t.Fatalf stops the test immediately.

The test database runs on a separate cluster. See postgres / dev test clusters.

Shared rows

Each test runs in its own transaction, and the transaction rolls back at the end. So the tests run in parallel against one database. That isolates what a test writes. It does not isolate a row that every test can see, such as a seeded user.

An insert that references a row through a foreign key takes a key-share lock on that row until its transaction ends. A seeded row that many fixtures point at can hold locks from many parallel transactions at once.

One of my tests ran UPDATE users SET team = NULL to hide the seeded users from the query under test. The update wrote the one seeded user while about twenty parallel transactions held key-share locks on it. Postgres failed it with an internal error: new multixact has more than one updating member. The seeded user was already off the team, so the update changed nothing the test read, and I deleted it.

A test does not write a row it did not insert. To keep seeded rows out of a result, the test filters on its own fixtures.

Committed rows

One test must commit. It tests a trigger that sends NOTIFY, and Postgres delivers a notification only when the transaction commits. Its rows stand for about a second, and the package test binaries run at the same time. A test in another package that read the jobs table with no filter of its own read them. It scanned a NULL into an *int64, or counted 1 where it wanted 0. Two tests failed on pushes that touched neither, and passed on a rerun.

A failure that nobody trusts is a failure that nobody reads. So each read of jobs in a test now names its own rows: by queue, by job name, or by an id that its own fixture made. A read matches an id in args by JSONB containment, not by a cast:

SELECT count(*) FROM jobs WHERE args @> jsonb_build_object('company_id', $1::bigint);

A cast such as (args->>'company_id')::bigint reads every row in the table. A row from another test with a non-numeric id then raises an error.

A test in the jobs package parses every other test file and fails on a read of jobs whose WHERE clause names none of the three. An allow list takes an exception, with one reason for each directory.

Tables

I write a table test when cases differ only in their inputs and wants. The table holds only the differences. A case with its own setup or its own output stays a separate test.

Every row calls t.Parallel():

for _, tc := range cases {
	t.Run(tc.name, func(t *testing.T) {
		t.Parallel()
		is := is.New(t)
		db := test.NewDB(t)
		// ...
	})
}

A row is a new test. Folding parallel tests into rows that are not parallel trades away concurrency that the suite needs as it grows.

Races

A test of http.TimeoutHandler used a 1 ms timeout. Its handler returned when the request context was cancelled. That is the same moment the timer fires. TimeoutHandler selects on the handler being done and on the context being done. So both cases were ready, and select picked one at random. On a loaded CI runner, the test read 200, not 503.

Now the handler blocks until the test ends, so the timer is the only outcome:

done := make(chan struct{})
t.Cleanup(func() { close(done) })
h := http.TimeoutHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	<-done
}), time.Millisecond, "timeout")

Cache

To reuse the go test cache across CI runs, I persist the cache directories and make each test binary depend only on its source.

Go keeps two on-disk caches:

On a CI VM I put both on the local disk and reuse them across runs:

export GOCACHE=/var/cache/ci/go-cache
export GOMODCACHE=/var/cache/ci/go-mod

The result cache keys on the inputs to each test binary. Two defaults change those inputs on every CI run:

Two flags remove both:

go test -buildvcs=false -trimpath ./...

Identical source then produces an identical binary, and go test reuses cached results across commits and across boxes that share a GOCACHE.

The first run populates the cache. The next run on the same source reports (cached) for unchanged packages and finishes in seconds. cmd / sockeye runs its CI cache this way.

A cached result includes what the test binary printed, and go test replays it on a hit. So a check that reads the run's output costs nothing when nothing changed. My template coverage check works this way. Each render prints a line to stdout. A command reads the lines back out of the -v output. See go / html templates.

The cache also keys on the environment variables the binary read. So a run with the trace on and a run with it off are separate entries.

← All articles