Level Up With Testcontainers

Source

The source code used throughout this series can be found here: https://github.com/anthoturc/testcontainers-w-go/.

Note: The testcontainers site does have a guide on this topic too! This post is inspired by their blog with a spin on how I would do the testing.

Table Of Contents

Open Table Of Contents

What? Testcontainers?

That’s not a typo. Testcontainers is an awesome piece of technology that encourages using real dependencies instead of mocking them in integration tests, end-to-end tests, and unit tests! Here is the official site.

This isn’t a particularly new piece of technology but it is an incredibly useful tool to have in the back pocket. If you navigate to the Testcontainers site, you will be met with:

Unit tests with real dependencies

So Testcontainers is a collection of libraries, written in different languages, that make it easy to spin up containers where your dependencies would go in production.

Why Testcontainers?

Depending on your background, you might be comfortable writing unit tests by mocking all your dependencies. That makes things easy because you are focused on your business logic. However, there is value in having a real dependency to test against. For example, if you are working with a Postgres instance in prod, you don’t actually know that your business logic will work against the real instance because you mocked everything.

Opinion incoming

While mocking is useful, it isn’t a silver bullet. If you can reasonably use a real dependency for your testing, you should do it.

Test Against Postgres

Let’s walk through how I would use Testcontainers in a toy example. In this example, I have written PingService. Its responsibility is to store IP Addresses passed to it in a Postgres table. If the IP already exists, then an error is returned.

I will be using the pgx driver and the database/sql package to interact with the DB.

Project Setup

I’ll start by creating a directory, and initializing a Go module. Don’t forget to change the module name to something that makes sense.

mkdir testcontainers-w-go
cd testcontainers-w-go
go mod init github.com/anthoturc/testcontainers-w-go

My .sql script will live in in testcontainers-w-go/db/ips.sql:

CREATE TABLE ips (
  id SERIAL PRIMARY KEY,
  ip_addr TEXT UNIQUE NOT NULL
);

In the project root I have a service.go

package main

import (
	"database/sql"
	"errors"
	"fmt"

	"github.com/jackc/pgconn"
	"github.com/jackc/pgerrcode"
)

type PingService struct {
	DB *sql.DB
}

type Ping struct {
	Id     int
	IpAddr string
}

var ErrIpAlreadyExists error = errors.New("ip address already exists")

// Ping will attempt to insert the supplied IP Address into the ips table.
// If the IP Address is already in the table, it will return an ErrIpAlreadyExists
// error.
// All other errors are wrapped and returned.
func (ps *PingService) Ping(ipAddr string) (*Ping, error) {
  row := ps.DB.QueryRow(
    `INSERT INTO ips (ip_addr)
    VALUES ($1) RETURNING id`, ipAddr)
  ping := &Ping{
    IpAddr: ipAddr,
  }
  err := row.Scan(&ping.Id)

  if err != nil {
    var e *pgconn.PgError
    if errors.As(err, &e) && e.Code == pgerrcode.UniqueViolation {
      return nil, ErrIpAlreadyExists
    }

    return nil, fmt.Errorf("create ping entry: %w", err)
  }

  return ping, nil
}

There is very little business logic here and most of the “hard work” is left to the database. If we wanted to test the Ping function we would probably want to test 1) inserting an IP Address and 2) inserting a duplicate. But I don’t want to mock the DB dependency of the PingService type because the uniqueness constraint is built into the definition of the table and mocking that would be a pain:

 ip_addr TEXT UNIQUE NOT NULL

Testconatiners, can help us here.

Setup Testcontainers and Test Cases

If you follow the Testcontainers guide, you can install the dependency to your project:

go get github.com/testcontainers/testcontainers-go
go get github.com/testcontainers/testcontainers-go/modules/postgres
go get github.com/stretchr/testify

Now, we want to test the ping service so we should set that up in service_test.go:

package main

import "testing"

func TestPing(t *testing.T) {
}

func TestPingWithIpAlreadyExists(t *testing.T) {
}

In production, we would want to limit the number of open connections to the database. In our tests, we can do the same thing by creating a single instance of the Postgres database and a single *sql.DB connection. These resources will be resused in each test. This is where I would differ from the Testcontainers article a little bit more. Instead of setting up a suite (via another dependency), I would just use a TestMain to perform setup and cleanup.

This was adapted from the Testcontainers quickstart example:

// -- snip --

var db *sql.DB // this is global!

// createPgContainer will standup a Postgres testcontainer for use across all tests
func createPgContainer(pgConf *PgConf) (*postgres.PostgresContainer, error) {
  container, err := postgres.RunContainer(context.Background(),
    testcontainers.WithImage("docker.io/postgres:15.2-alpine"),
    postgres.WithInitScripts(filepath.Join("db", "ips.sql")),
    postgres.WithDatabase(pgConf.Database),
    postgres.WithPassword(pgConf.Password),
    postgres.WithUsername(pgConf.UserName),
    postgres.WithDatabase(pgConf.Database),
    testcontainers.WithWaitStrategy(
      wait.ForLog("database system is ready to accept connections").
        WithOccurrence(2).
        WithStartupTimeout(5*time.Second)),
  )
  return container, err
}

You don’t need to worry about the exact definition of PgConf. If you are curious, feel free to checkout the GitHub repo. It holds a little data that is useful for connecting to Postgres.

Now let’s setup TestMain:

// -- snip --

// TestMain will setup the dependencies needed for all tests
func TestMain(m *testing.M) {
  pgConf := DefaultPgConf()
  container, err := createPgContainer(pgConf)
  if err != nil {
    log.Fatalf("unable to start container: %+v", err)
  }

  defer func() {
    if err := container.Terminate(context.Background()); err != nil {
      log.Fatalf("unable to stop container: %+v", err)
    }
  }()

  connStr, err := container.ConnectionString(
      context.Background(),
      fmt.Sprintf("sslmode=%s", pgConf.SSLMode),
      fmt.Sprintf("dbname=%s", pgConf.Database))
  if err != nil {
    log.Fatalf("Failed to connect to Pg %s", connStr)
  }

  // Open is a wrapper around sql.Open
  db, err = Open(connStr)
  if err != nil {
    log.Fatalf("couldn't open with %s", connStr)
  }
  defer db.Close()

  code := m.Run()

  os.Exit(code)
}

There are a few things worth noting in this setup. Each line in this function gets run a single time. So we didn’t need to pull in the sync package and use sync.Once to get the container and DB conneciton established. The container has a ConnectionString method that you can pass options into. In this case, you might not want SSL enabled for your unit tests. The conatiner cleanup and db.Close() are done via defer.

Updating The Tests

Let’s start with the positive test case:

// -- snip --
func TestPing(t *testing.T) {
  ps := &PingService{
    DB: db,
  }

  ipAddr := "12.123.1.1"
  ping, err := ps.Ping(ipAddr)
  assert.NoError(t, err)

  assert.NotNil(t, ping)
  assert.NotNil(t, ping.IpAddr)
  assert.Equal(t, ipAddr, ping.IpAddr)
  assert.True(t, ping.Id > -1)
}

Awesome! We don’t need to validate whether a dependency was interacted with X times or whether specific arguments were supplied to the DB dependency. The negative test case, will pretty much look the same but there are a few other checks to perform.

// -- snip --
func TestPingWithIpAlreadyExists(t *testing.T) {
	ps := &PingService{
		DB: db,
	}

	ipAddr := "123.123.123.1"
	_, err := ps.Ping(ipAddr)
  assert.NoError(t, err)

	_, err = ps.Ping(ipAddr)
	assert.Error(t, err, "There m be an error!")
	assert.ErrorAs(t, err, &ErrIpAlreadyExists, "The error should be an ErrIpAlreadyExists")
}

Run the Tests

Testcontainers doesn’t need any extra setup. The go test tool can be used the same way you are familiar with!

go test -v ./...
...
10:15:50 🐳 Creating container for image testcontainers/ryuk:0.6.0
10:15:50 ✅ Container created: e9d553c0d1ae
10:15:50 🐳 Starting container: e9d553c0d1ae
10:15:50 ✅ Container started: e9d553c0d1ae
10:15:50 🚧 Waiting for container id e9d553c0d1ae image: testcontainers/ryuk:0.6.0. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
10:15:50 🔔 Container is ready: e9d553c0d1ae
10:15:50 🐳 Creating container for image docker.io/postgres:15.2-alpine
10:15:50 ✅ Container created: 3b218df6a7fe
10:15:51 🐳 Starting container: 3b218df6a7fe
10:15:51 ✅ Container started: 3b218df6a7fe
10:15:51 🚧 Waiting for container id 3b218df6a7fe image: docker.io/postgres:15.2-alpine. Waiting for: &{timeout:<nil> deadline:0xc000014b58 Strategies:[0xc000573800]}
10:15:53 🔔 Container is ready: 3b218df6a7fe
=== RUN   TestPing
--- PASS: TestPing (0.04s)
=== RUN   TestPingWithIpAlreadyExists
--- PASS: TestPingWithIpAlreadyExists (0.00s)
PASS
ok  	github.com/anthoturc/testcontainers-w-go	9.608s

Conclusion

In this post, I covered what Testcontainers are and why you would want to use them. Again, this article was inspired by this Testcontainers blog but here are the main differences:

In this case, these were all unit tests but there is nothing stopping us from using Testcontainers to write full blown integration tests or end-to-end tests with similar setup!