web / caddy

When I run a web service on a VM, I use Caddy as a reverse proxy to terminate TLS in front of the app.

Config

# /etc/caddy/Caddyfile
sockeye.example.com {
  reverse_proxy 127.0.0.1:1994
}

Point DNS at the box and reload:

sudo cp Caddyfile /etc/caddy/Caddyfile
sudo systemctl reload caddy

The app listens on localhost. Caddy renews a Let's Encrypt certificate.

Hold requests during a restart

The sockeye server runs as one process, not as two slots behind a switch. A deploy restarts that process, and a restart takes about a second. Caddy retries the upstream for that time, so a request waits and does not get a 502:

sockeye.example.com {
  reverse_proxy 127.0.0.1:1994 {
    lb_try_duration 30s
    lb_try_interval 250ms
  }
}

A reader sees a pause, and a client such as a git push or a long poll continues. The same rule covers a crash, which systemd restarts after RestartSec. Caddy retries only when it cannot connect to the process. A request that the process accepted waits for the process to answer. So the server bounds its own waits, such as a query on a dead database connection. A deploy that fails its health check puts the previous binary back and restarts again, behind the same retry.

The host has one CPU, so a second slot would compete with the first for it. One process keeps the deploy short. It does not keep the site up across a host failure.

Access log

Caddy writes one access line per request to the journal. The sockeye runners ping the server each second, and my services post perf records every few seconds. Those lines would fill most of the journal, so Caddy skips them:

sockeye.example.com {
  log
  @ping path /box-ping
  log_skip @ping
  @ingest {
    method POST
    path /perf/*
  }
  log_skip @ingest
  reverse_proxy 127.0.0.1:1994
}

The server's own request log keeps each perf post. The journal then holds 7 days of the other lines (see unix / systemd).

Compared to a PaaS

A PaaS like Render provisions certificates and terminates HTTPS at a shared edge with DDoS protection. Caddy gives me the same certificates on a VM. I lose the edge and gain control and a flat cost.

Caddy vs nginx

Both are fast enough at my scale. Caddy renews certificates by default. nginx needs certbot and a renewal timer. Caddy is a single Go binary, like everything else I deploy.

Caddy vs an app server

Caddy runs in front of an application server:

client --HTTPS--> Caddy (:443, TLS) --HTTP--> app (127.0.0.1)

Caddy fronts any localhost listener: a Go binary, or an app server such as Puma for a Ruby app.

CAA records

If certificate issuance fails, check the domain's CAA records. They restrict which authorities can issue certificates. If the apex excludes Let's Encrypt, add a record for the subdomain:

sockeye  CAA  0 issue "letsencrypt.org"

A subdomain inherits CAA records until a closer record exists. This record overrides the apex policy for that host.

← All articles