cmd / sockeye
sockeye is a git forge that I run on my own VMs. It hosts private git
repos, runs their checks, and merges changes. Its site,
sockeye.sh, has the install steps and the CLI
reference.
- One binary,
soc, is the server, the runner, and the CLI. - One Postgres database holds the state. Bare git repos hold the code.
- A check starts within 1 second of a push.
- A runner pings every second. After 15 seconds of silence, the server gives its job to another runner.
Why run my own
An instance serves one organization without multi-tenant abstractions or billing tiers. The instance satisfies my specific requirements. When I encounter limits, I modify the code directly. The change page displays patches up to 2 MiB inline.
Architecture
The server is an HTTP server. It owns the repos, serves the dashboard and the git transport, creates jobs from pushes, and merges changes. I run one server. It serializes merges and holds live job output in memory.
A runner long polls the server for jobs. For each job it checks out the
job's tree, finds the Checkfile in the job's directory, and runs the
command. It reports success, failure, or error to the server.
soc server
soc runner
I host the server on a small Ubicloud VM and the runners on a larger one. The runners share Go's build and module caches on local disk.
Self-hosted git
Each repo is a bare repo at /srv/git/<name>.git. The server serves
the git transport itself, and Caddy proxies /git to it
for TLS. For each request the server checks the token, then runs
git http-backend over the repo. It spools a push to a temp file
first, so the backend gets a length for a push of any size.
There is no ssh git transport. An ssh key only signs the login.
On each machine, I point the CLI at the server, then log in:
soc git setup https://sockeye.example.com/git
soc login
soc login signs a challenge with a key from my ssh-agent and trades
the signature for a token. The token lives seven days in
~/.config/sockeye/token, and git reads it through
soc auth git-credential. soc login also prints a five-minute link
that signs my browser in.
The server is origin, and the only remote. A pre-receive hook
rejects direct writes to main. A post-receive hook reports each push
to the server.
Push
The hook POSTs the refs that moved. The server then:
- Reads the changed files out of the bare repo.
- Collects the directories that contain those files.
- Walks up from each directory to the root.
- Reads the
Checkfileat each level. - Creates a job for each entry.
A change to sdk/go/account.go runs the Checkfile in sdk/go/,
sdk/, and /. Added, removed, renamed, and modified files all count.
A push, a finished job, or a new runner calls one loop on the server. In one transaction, the loop pairs an unassigned job with an idle runner. It repeats until no job or no runner is left.
Checkfile
A Checkfile defines the checks for its directory. A check is any
command that passes or fails. Each line has a name and a command,
separated by a colon:
gotest: go test -race -cover ./...
lint: test -z "$(go tool goimports -l .)"
The name appears as a check on the change page. A runner runs the
command in /bin/bash -eo pipefail. A comment starts with #.
A repo pins its own goimports and deadcode, so a check runs the
version the repo chose (see go / checks).
A Checkfile can also name one artifact, a file that a build leaves:
build: go build -o server ./cmd/server && tar czf deploy.tar.gz server web
artifact: deploy.tar.gz
The artifact line runs nothing. When a check passes and the file
exists, the runner uploads it. The server keeps it in object storage
under the hash of the commit's tree, not the commit's SHA. The squash
commit that a merge writes has a new SHA but the same tree. So the
deploy hook finds the file and installs the bytes that CI tested. A
tree with no artifact deploys from source.
Changes
A change is the review unit. It has an id like APP-42. The id is
also its branch name and its worktree directory name.
soc checkout with no id allocates a change. The server takes the next
number and writes the branch at main. The CLI cuts a worktree at
~/.worktrees/<repo>/<ID> and prints its path.
It also symlinks an untracked .env from the main clone, then runs the
command that the clone's tree.setup git config key names:
git config tree.setup bin/tree-setup
One repo uses it to give each worktree its own development database (see postgres / dev test clusters). The key is config, not a tracked hook, so checking out somebody else's change cannot run their branch.
I write the title and description with soc edit before the work, so
the dashboard says what each open change is. The first push of a branch
with one commit replaces both with that commit's subject and body. A
later push leaves them alone.
Review
A review is one comment from the terminal, with no inline comment, no threading, and no approval state. Each action has one way to do it, so an agent does it the same way every time:
soc show APP-42
git fetch origin && git diff origin/main...origin/APP-42
echo "the retry loop needs a ceiling" | soc comment APP-42
The server refuses a comment when the reviewer's commit is behind the change's head.
The web UI shows the state. Its only forms revoke a token and activate or deactivate an account. For each other write, it shows the command to run. A write wakes every open page, and each live section fetches itself again (see go / wakeups and web / live regions).
Merge
soc push --wait
soc merge
soc push --wait pushes the change branch and blocks until its checks finish.
soc merge tells the server to squash-merge the change onto main.
The server checks that the change has a title, that it merges cleanly,
and that all checks on its head passed. An instance can also require a
title pattern.
The server writes a squash commit with Co-Authored-By and
Reviewed-by trailers, updates main, and backs the commit up (see
mirrors and backups). If the backup fails,
main stays where it was.
A repo can also deploy on merge. A hook on the server takes the merged
SHA and ships it, and soc merge prints the hook's output.
Other repos deploy from a laptop and record each deploy with sockeye.
soc deploy live shows, for each service, the live SHA and the changes
in it. A change is live when its merge commit is an ancestor of that
SHA.
Login
A server with WorkOS SSO authenticates users through the organization's
identity provider. A soc login link prompts for an authenticator code.
A runner mints its token using its host SSH key. The server grants read-only access to all repositories. The runner refreshes its token every six days. The server journal logs logins, credential updates, merges, and repo operations.
Several instances
I maintain isolated instances for personal projects and work. They share no virtual machines, databases, or keys.
Inside a clone, the CLI defaults to origin. Outside a repository,
SOCKEYE_HOST sets the destination server:
SOCKEYE_HOST=sockeye.example.com soc login
Output
The server stores check output as plain text for 30 days. A check reports
failed on a non-zero exit and errored on a runner setup failure.
soc check APP-42 sdk/go/tests
Logs and performance
A service names its systemd units with soc service units. Its logs
page then shows its journal: a live tail, a time range, and a search.
The search reads key=value fields from logfmt and JSON lines, so
status>=500 path=/login finds the failed logins. soc logs takes
the same terms.
The server cannot read the journal itself. A root relay reads it for the units in an allowlist (see unix / systemd).
The perf page reads the request lines in the same journal. It draws requests by status class, the p50, p95, and p99 response time, and a table of routes, with a marker for each deploy. Hourly histograms keep 400 days. A click on a chart opens the log lines for that span, and a drag zooms (see web / charts).
The jobs page reads one line from each scheduled job, such as
job=ingest status=ok duration=41.4s. It draws the runs by status and
the p95 duration, with a table of jobs. A job is a systemd timer (see
unix / systemd), and its service joins the
list in soc service units.
Errors
A service posts its errors to the server with a key of its own. The
server groups the events by fingerprint, or by type and top frames.
/errors and soc error show the groups.
soc error list # open error groups
soc error show <ID> # events, extras, and stack trace
show prints the error group, then one line for each of its newest
events, with the release and extras. The stack trace comes last,
newest frame first.
A commit message with Fixes: err-12 resolves that group when the
change merges. An event from a later release that holds the fix opens
the group again.
Mirrors and backups
A repo that needs GitHub, for go get or for a deploy platform such as
Render, gets a mirror of main there, and the
server pushes each squash commit to it. For every other repo, the server
puts a git bundle of each squash commit in object storage.
I develop hml,
highlight, is, and
errs on sockeye and mirror them to GitHub, which
go get resolves. GitHub receives main and the tags. sockeye
never mirrors change branches, so I close pull requests there. Each
README says so.