cmd / sockeye

sockeye is a git forge that I run on my own VMs. It hosts private git repos, runs their checks, and merges changes. Its site, sockeye.sh, has the install steps and the CLI reference.

Why run my own

An instance serves one organization without multi-tenant abstractions or billing tiers. The instance satisfies my specific requirements. When I encounter limits, I modify the code directly. The change page displays patches up to 2 MiB inline.

Architecture

The server is an HTTP server. It owns the repos, serves the dashboard and the git transport, creates jobs from pushes, and merges changes. I run one server. It serializes merges and holds live job output in memory.

A runner long polls the server for jobs. For each job it checks out the job's tree, finds the Checkfile in the job's directory, and runs the command. It reports success, failure, or error to the server.

soc server
soc runner

I host the server on a small Ubicloud VM and the runners on a larger one. The runners share Go's build and module caches on local disk.

Self-hosted git

Each repo is a bare repo at /srv/git/<name>.git. The server serves the git transport itself, and Caddy proxies /git to it for TLS. For each request the server checks the token, then runs git http-backend over the repo. It spools a push to a temp file first, so the backend gets a length for a push of any size.

There is no ssh git transport. An ssh key only signs the login.

On each machine, I point the CLI at the server, then log in:

soc git setup https://sockeye.example.com/git
soc login

soc login signs a challenge with a key from my ssh-agent and trades the signature for a token. The token lives seven days in ~/.config/sockeye/token, and git reads it through soc auth git-credential. soc login also prints a five-minute link that signs my browser in.

The server is origin, and the only remote. A pre-receive hook rejects direct writes to main. A post-receive hook reports each push to the server.

Push

The hook POSTs the refs that moved. The server then:

  1. Reads the changed files out of the bare repo.
  2. Collects the directories that contain those files.
  3. Walks up from each directory to the root.
  4. Reads the Checkfile at each level.
  5. Creates a job for each entry.

A change to sdk/go/account.go runs the Checkfile in sdk/go/, sdk/, and /. Added, removed, renamed, and modified files all count.

A push, a finished job, or a new runner calls one loop on the server. In one transaction, the loop pairs an unassigned job with an idle runner. It repeats until no job or no runner is left.

Checkfile

A Checkfile defines the checks for its directory. A check is any command that passes or fails. Each line has a name and a command, separated by a colon:

gotest: go test -race -cover ./...
lint: test -z "$(go tool goimports -l .)"

The name appears as a check on the change page. A runner runs the command in /bin/bash -eo pipefail. A comment starts with #.

A repo pins its own goimports and deadcode, so a check runs the version the repo chose (see go / checks).

A Checkfile can also name one artifact, a file that a build leaves:

build: go build -o server ./cmd/server && tar czf deploy.tar.gz server web
artifact: deploy.tar.gz

The artifact line runs nothing. When a check passes and the file exists, the runner uploads it. The server keeps it in object storage under the hash of the commit's tree, not the commit's SHA. The squash commit that a merge writes has a new SHA but the same tree. So the deploy hook finds the file and installs the bytes that CI tested. A tree with no artifact deploys from source.

Changes

A change is the review unit. It has an id like APP-42. The id is also its branch name and its worktree directory name.

soc checkout with no id allocates a change. The server takes the next number and writes the branch at main. The CLI cuts a worktree at ~/.worktrees/<repo>/<ID> and prints its path.

It also symlinks an untracked .env from the main clone, then runs the command that the clone's tree.setup git config key names:

git config tree.setup bin/tree-setup

One repo uses it to give each worktree its own development database (see postgres / dev test clusters). The key is config, not a tracked hook, so checking out somebody else's change cannot run their branch.

I write the title and description with soc edit before the work, so the dashboard says what each open change is. The first push of a branch with one commit replaces both with that commit's subject and body. A later push leaves them alone.

Review

A review is one comment from the terminal, with no inline comment, no threading, and no approval state. Each action has one way to do it, so an agent does it the same way every time:

soc show APP-42
git fetch origin && git diff origin/main...origin/APP-42
echo "the retry loop needs a ceiling" | soc comment APP-42

The server refuses a comment when the reviewer's commit is behind the change's head.

The web UI shows the state. Its only forms revoke a token and activate or deactivate an account. For each other write, it shows the command to run. A write wakes every open page, and each live section fetches itself again (see go / wakeups and web / live regions).

Merge

soc push --wait
soc merge

soc push --wait pushes the change branch and blocks until its checks finish. soc merge tells the server to squash-merge the change onto main. The server checks that the change has a title, that it merges cleanly, and that all checks on its head passed. An instance can also require a title pattern.

The server writes a squash commit with Co-Authored-By and Reviewed-by trailers, updates main, and backs the commit up (see mirrors and backups). If the backup fails, main stays where it was.

A repo can also deploy on merge. A hook on the server takes the merged SHA and ships it, and soc merge prints the hook's output.

Other repos deploy from a laptop and record each deploy with sockeye. soc deploy live shows, for each service, the live SHA and the changes in it. A change is live when its merge commit is an ancestor of that SHA.

Login

A server with WorkOS SSO authenticates users through the organization's identity provider. A soc login link prompts for an authenticator code.

A runner mints its token using its host SSH key. The server grants read-only access to all repositories. The runner refreshes its token every six days. The server journal logs logins, credential updates, merges, and repo operations.

Several instances

I maintain isolated instances for personal projects and work. They share no virtual machines, databases, or keys.

Inside a clone, the CLI defaults to origin. Outside a repository, SOCKEYE_HOST sets the destination server:

SOCKEYE_HOST=sockeye.example.com soc login

Output

The server stores check output as plain text for 30 days. A check reports failed on a non-zero exit and errored on a runner setup failure.

soc check APP-42 sdk/go/tests

Logs and performance

A service names its systemd units with soc service units. Its logs page then shows its journal: a live tail, a time range, and a search. The search reads key=value fields from logfmt and JSON lines, so status>=500 path=/login finds the failed logins. soc logs takes the same terms.

The server cannot read the journal itself. A root relay reads it for the units in an allowlist (see unix / systemd).

The perf page reads the request lines in the same journal. It draws requests by status class, the p50, p95, and p99 response time, and a table of routes, with a marker for each deploy. Hourly histograms keep 400 days. A click on a chart opens the log lines for that span, and a drag zooms (see web / charts).

The jobs page reads one line from each scheduled job, such as job=ingest status=ok duration=41.4s. It draws the runs by status and the p95 duration, with a table of jobs. A job is a systemd timer (see unix / systemd), and its service joins the list in soc service units.

Errors

A service posts its errors to the server with a key of its own. The server groups the events by fingerprint, or by type and top frames. /errors and soc error show the groups.

soc error list           # open error groups
soc error show <ID>      # events, extras, and stack trace

show prints the error group, then one line for each of its newest events, with the release and extras. The stack trace comes last, newest frame first.

A commit message with Fixes: err-12 resolves that group when the change merges. An event from a later release that holds the fix opens the group again.

Mirrors and backups

A repo that needs GitHub, for go get or for a deploy platform such as Render, gets a mirror of main there, and the server pushes each squash commit to it. For every other repo, the server puts a git bundle of each squash commit in object storage.

I develop hml, highlight, is, and errs on sockeye and mirror them to GitHub, which go get resolves. GitHub receives main and the tags. sockeye never mirrors change branches, so I close pull requests there. Each README says so.

← All articles