go / http clients

I prefer a small client of my own to a vendor SDK.

The standard library net/http covers most of what an SDK wraps:

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
)

func main() {
	url := os.Getenv("API_URL")
	if url == "" {
		log.Fatalln("err: API_URL environment variable is not set")
	}

	reqBody, err := json.Marshal(map[string]string{"text": "hi"})
	if err != nil {
		log.Fatalf("err: %v\n", err)
	}

	resp, err := http.Post(url, "application/json", bytes.NewBuffer(reqBody))
	if err != nil {
		log.Fatalf("err: %v\n", err)
	}
	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatalf("err: %v\n", err)
	}

	fmt.Println(string(body))
}

Ruby's net/http has a clumsy interface, so I used the http gem there. The same call is two lines, plus a dependency:

resp = HTTP.post(ENV.fetch("API_URL"), json: {text: "hi"})
puts resp.body.to_s

Go's standard library is good enough that I add no package.

SDK costs

An SDK needs upgrades for security patches and dependency conflicts. It can become unmaintained. It is one more interface to learn.

My own client covers only the endpoints the app calls. Most of an SDK is code the app never uses.

Shared shape

All my clients share one helper for retries, so a client is mostly request building and response mapping:

res, err := httputil.Do(ctx, func() (*http.Request, error) {
	req, err := http.NewRequestWithContext(ctx, method, url, body)
	if err != nil {
		return nil, err
	}
	req.Header.Set("Authorization", "Bearer "+c.apiKey)
	return req, nil
}, httputil.Config{
	Client:         c.http,
	RetryDelays:    retryDelays,
	TransientCodes: transientCodes,
})

httputil.Do calls the build closure once per attempt. The retry schedule is a lookup table. Each client picks its own transient codes and delays.

Tests point the HTTP client at an httptest.NewServer. Higher layers take one-method interfaces such as Uploader and inject fakes.

Narrow responses

A response struct holds only the fields a caller reads. A vendor's person record can have a hundred fields. If my code reads six, the struct has six, and encoding/json ignores the rest.

A field that nothing reads is still code. It is a name that a reader checks for callers, and a type that breaks when the vendor changes it. I went through six vendor clients and deleted each field with no reader. One client held a full employment history that another vendor's data had replaced.

I decode a nullable vendor object straight into its zero value, not through a pointer envelope:

type Organization struct {
	Name   string `json:"name"`
	Domain string `json:"primary_domain"`
}

type personResponse struct {
	Person struct {
		Title        string       `json:"title"`
		Organization Organization `json:"organization"`
	} `json:"person"`
}

A JSON null leaves the struct at its zero value, so the caller checks Organization.Domain == "" once. A *Organization makes every caller check nil first, and a missed check panics.

Clients

I built these clients this way:

Example: Slack

A client for Slack's Web API shows the shape on three methods: chat.postMessage, chat.update, and chat.delete. Two rules come from the vendor, and the client puts each one in one place.

Retry policy belongs to the method. Post and Update are not idempotent, and a retried post can duplicate a message. Both run once. Delete is idempotent, so it retries on the default schedule. The schedule is an argument to the shared do, so the policy sits at the call site:

func (c *Client) Delete(ctx context.Context, channel, ts string) error {
	body := map[string]any{"channel": channel, "ts": ts}
	_, err := c.do(ctx, c.deleteURL, body, httputil.DefaultRetryDelays)
	return err
}

Slack returns HTTP 200 for application failures such as channel_not_found, with {"ok": false, "error": "..."} in the body. do checks the status and the ok field, and it returns the raw body when parsing fails:

if res.StatusCode/100 != 2 {
	return apiResponse{}, fmt.Errorf("slack: HTTP %d", res.StatusCode)
}
var parsed apiResponse
if err := json.Unmarshal(res.Body, &parsed); err != nil {
	return apiResponse{}, fmt.Errorf("slack: parse response: %s", string(res.Body))
}
if !parsed.OK {
	return apiResponse{}, fmt.Errorf("slack: %s", parsed.Error)
}

A Block Kit block is a map[string]any, so callers stay close to the JSON in Slack's Block Kit Builder. The cost is no compile-time schema check. A worker that only posts takes a one-method Poster interface, and tests inject a fake.

Example: Postmark inbound webhook

Postmark posts inbound email to a webhook, so the handler is a server that takes untrusted input from a client. It needs two layers of authentication and one guard against retries.

Basic auth covers the transport. The comparison takes constant time, and a missing credential is an empty string, so a malformed request gets the same 401 as a wrong password:

func basicAuthOK(r *http.Request, wantUser, wantPass string) bool {
	user, pass, _ := r.BasicAuth()
	u := subtle.ConstantTimeCompare([]byte(user), []byte(wantUser))
	p := subtle.ConstantTimeCompare([]byte(pass), []byte(wantPass))
	return u&p == 1
}

My Ruby handler did the same with Rack::Utils.secure_compare, after coercing both sides with .to_s.

DKIM authenticates the message. The handler reads the Authentication-Results headers and requires a pass for my own domain:

var dkimPass = regexp.MustCompile(`(?i)\bdkim=pass\b.*\b(?:header\.)?d=example\.com\b`)

Postmark retries a failed webhook up to 10 times. The handler inserts each message ID into a table with a unique index. A duplicate gets a 200 and no second run of the work, so the retries stop.

Libraries I do use

A narrow, audited library for one job, such as JWT verification, is a fair trade. I avoid the vendor SDK that pulls a large dependency tree to wrap endpoints I can call in a dozen lines.

← All articles