go / syntax highlighting
highlight colors source code for server-rendered HTML. It has a hand-written, line-oriented scanner per language over a shared token type.
highlight.Code(w, "main.go", src) // or "go", from a markdown fence
highlight.Diff("main.go", patch) // a unified-diff patchIt colors this blog's code fences, built by cmd / blog, and the diff view in cmd / cibot.
Scope
Scanners exist for Go, SQL, CSS, JavaScript, C, Ruby, JSON, Lua, HTML, Markdown, shell, and hml. A format with no scanner comes out escaped and uncolored. A word a keyword list misses comes out uncolored.
This blog ran Chroma before. Chroma carries hundreds of lexers and a transitive regexp2, and it had no lexer for hml. I had already written the scanners in cibot for its diff pages.
Classes
The output uses a short list of classes: k keywords, n names, s
strings, m numbers, c comments. A diff row carries one more: gi
inserted, gd deleted, gu hunk header, gc context.
There is no shipped palette. The consuming site styles the classes. This blog styles them twice: a light palette and Catppuccin Frappe, the flavor my editor uses.
Two layers
Tokens are the lower layer. A scanFunc reads one line, takes the
state the previous line ended in, and returns the state this one ends
in. A lexer restarted per line colors the second half of a raw string
as code.
Emitters are the upper layer. Code writes a whole file. Diff writes
a unified-diff patch, and each row also carries its diff class.
Both emitters HTML-escape every byte they write. In cibot the source is a file a stranger pushed.
Line orientation
A patch is a set of hunks with gaps, so there is no whole file to lex. The scanner colors each row from the state the row above it left. Rows the patch skipped leave that state approximate, which is acceptable in a diff. A whole file has no gap, so its state is exact.