cmd / sockeye

sockeye is a git forge that I run on my own VMs. It hosts private git repos, runs their checks, and merges changes. Its site, sockeye.sh, has the install steps and the CLI reference.

Why run my own

An instance serves one organization. It has no tenants, plans, or billing. A shared service carries the needs of many customers, and each feature has to suit all of them. My instance carries only mine, and I change it when I hit a limit.

On October 5, the change page showed "(diff too large to display)" for a 277 KB rewrite of a CSS file. I had to check out the branch to read it. I fixed the server the same day. A patch up to 2 MiB now loads when I open its row.

Architecture

The server is an HTTP server. It owns the repos, serves the dashboard and the git transport, creates jobs from pushes, and merges changes. I run one server. It serializes merges and holds live job output in memory.

A runner long polls the server for jobs. For each job it checks out the job's tree, finds the Checkfile in the job's directory, and runs the command. It reports success, failure, or error to the server.

soc server
soc runner

I host the server on a small Ubicloud VM and the runners on a larger one. The runners share Go's build and module caches on local disk.

Why SQLite

The database is one file on the server VM. There is no database server to run, patch, or firewall. The server opens it in WAL mode with one connection, so writes happen one at a time.

The server backs it up on its own schedule, with no cron job. Each hour it takes a snapshot with SQLite online backup, gzips it, and uploads it to object storage. Each week it restores the newest snapshot and checks database integrity to prove the backup works.

A managed Postgres database, such as Ubicloud's, keeps the backups and takes concurrent writers, and it costs more. For my projects, one file costs less.

Self-hosted git

Each repo is a bare repo at /srv/git/<name>.git. The server serves the git transport itself, and Caddy proxies /git to it for TLS. For each request the server checks the token, then runs git http-backend over the repo. It spools a push to a temp file first, so the backend gets a length for a push of any size.

There is no ssh git transport. An ssh key only signs the login.

On each machine, I point the CLI at the server, then log in:

soc git setup https://sockeye.example.com/git
soc login

soc login signs a challenge with a key from my ssh-agent and trades the signature for a token. The token lives seven days in ~/.config/sockeye/token, and git reads it through soc auth git-credential. soc login also prints a five-minute link that signs my browser in.

The server is origin, and the only remote. A pre-receive hook rejects direct writes to main. A post-receive hook reports each push to the server.

Push

The hook POSTs the refs that moved. The server then:

  1. Reads the changed files out of the bare repo.
  2. Collects the directories that contain those files.
  3. Walks up from each directory to the root.
  4. Reads the Checkfile at each level.
  5. Creates a job for each entry.

A change to sdk/go/account.go runs the Checkfile in sdk/go/, sdk/, and /. Added, removed, renamed, and modified files all count.

A push, a finished job, or a new runner calls one loop on the server. In a SQLite transaction, the loop pairs an unassigned job with an idle runner. It repeats until no job or no runner is left.

Checkfile

A Checkfile defines the checks for its directory. A check is any command that passes or fails. Each line has a name and a command, separated by a colon:

gotest: go test -race -cover ./...
lint: test -z "$(go tool goimports -l .)"

The name appears as a check on the change page. A runner runs the command in /bin/bash -eo pipefail. A comment starts with #.

A repo pins its own goimports and deadcode, so a check runs the version the repo chose (see go / checks).

A Checkfile can also name one artifact, a file that a build leaves:

build: go build -o server ./cmd/server && tar czf deploy.tar.gz server web
artifact: deploy.tar.gz

The artifact line runs nothing. When a check passes and the file exists, the runner uploads it. The server keeps it in object storage under the hash of the commit's tree, not the commit's SHA. The squash commit that a merge writes has a new SHA but the same tree. So the deploy hook finds the file and installs the bytes that CI tested. A tree with no artifact deploys from source.

Changes

A change is the review unit. It has an id like APP-42. The id is also its branch name and its worktree directory name.

soc checkout with no id allocates a change. The server takes the next number and writes the branch at main. The CLI cuts a worktree at ~/.worktrees/<repo>/<ID> and prints its path.

It also symlinks an untracked .env from the main clone, then runs the command that the clone's tree.setup git config key names:

git config tree.setup bin/tree-setup

One repo uses it to give each worktree its own development database (see postgres / dev test clusters). The key is config, not a tracked hook, so checking out somebody else's change cannot run their branch.

I write the title and description with soc edit before the work, so the dashboard says what each open change is. The first push of a branch with one commit replaces both with that commit's subject and body. A later push leaves them alone.

Review

A review is one comment from the terminal, with no inline comment, no threading, and no approval state. Each action has one way to do it, so an agent does it the same way every time:

soc show APP-42
git fetch origin && git diff origin/main...origin/APP-42
echo "the retry loop needs a ceiling" | soc comment APP-42

The server refuses a comment when the reviewer's commit is behind the change's head.

The web UI shows the state. Its only forms revoke a token and activate or deactivate an account. For each other write, it shows the command to run. A write wakes every open page, and each live section fetches itself again (see go / wakeups and web / live regions).

Merge

soc push --wait
soc merge

soc push --wait pushes the change branch and blocks until its checks finish. soc merge tells the server to squash-merge the change onto main. The server checks that the change has a title, that it merges cleanly, and that all checks on its head passed. An instance can also require a title pattern.

The server writes a squash commit with Co-Authored-By and Reviewed-by trailers, updates main, and backs the commit up (see mirrors and backups). If the backup fails, main stays where it was.

A repo can also deploy on merge. A hook on the server takes the merged SHA and ships it, and soc merge prints the hook's output.

Other repos deploy from a laptop and record each deploy with sockeye. soc deploy live shows, for each service, the live SHA and the changes in it. A change is live when its merge commit is an ancestor of that SHA.

Login

A server with WorkOS SSO signs people in through the company's identity provider, which checks the second factor. A soc login link asks for an authenticator code on every server.

A runner mints its own token the same way a laptop does. It signs a challenge with an ssh key on the runner VM. The server gives the runner's service account a token that reads every repo and writes none. The runner mints again when the token is six days old.

The server's journal is the audit log. It records logins, account and key changes, tokens, merges, and repo removals.

Several instances

I run one instance for personal projects and one for work. They share no VM, database, or key, so a machine with access to one cannot reach the other.

Inside a clone, the CLI uses the clone's origin. Outside one, SOCKEYE_HOST names the server:

SOCKEYE_HOST=sockeye.example.com soc login

Each instance labels its authenticator entry with its own host.

After a merge, I deploy both instances from my laptop. A deploy program boots the new build of the first in an idle slot and flips traffic to it. A script upgrades the second. Both migrate the database before they install the new binary.

Output

SQLite stores check output as plain text for 30 days. A check reports failed on a non-zero exit and errored on a runner setup failure.

soc check APP-42 sdk/go/tests

Mirrors and backups

A repo that needs GitHub, for go get or for a deploy platform such as Render, gets a mirror of main there, and the server pushes each squash commit to it. For every other repo, the server puts a git bundle of each squash commit in object storage.

I develop hml, highlight, and is on sockeye and mirror them to GitHub, which go get resolves. GitHub receives main and the tags. sockeye never mirrors change branches, so I close pull requests there. Each README says so.

← All articles