vim / tree-sitter grammar

updated August 05, 2026

hml is my own template language, so no editor knows it. Neovim highlights with tree-sitter, so I wrote a grammar.

It lives in the hml repo next to the Go engine: grammar.js, an external scanner in src/scanner.c, and queries under queries/.

Indentation needs a scanner

hml maps indentation to HTML nesting, so a block is a run of lines indented past the line that opened it. A context-free grammar cannot express that, which is what tree-sitter's external scanner is for. The scanner emits zero-width INDENT and DEDENT tokens at the end of a line and leaves the newline to the grammar's extras:

externals: ($) => [
  $._indent,
  $._dedent,
  $.filter_body,
  $.comment_body,
  $._error_sentinel,
],

_block is then ordinary:

_block: ($) => seq($._indent, repeat1($._item), $._dedent),

Zero width matters. The token ends where it started, so several DEDENTs can land at one position and a rescan from the same byte measures the same next line again.

The scanner keeps a stack of indent widths, at most 64 deep. Its whole state fits in the buffer tree-sitter passes, so serialize writes the struct whole and deserialize starts over at the top level when the length disagrees with what was written.

Tree-sitter sets _error_sentinel during error recovery. The scanner reads it and gives up, rather than inventing indentation inside a broken tree. A file mid-edit is the common case in an editor.

A filter or comment body is scanned the same way, as one opaque token covering every line indented past its opener. The engine hands those lines to JavaScript or CSS, or drops them, so neither is parsed.

The grammar mirrors the Go parser

The precedence table exists so ! binds looser than a comparison, the way expr.go orders parseOr, parseAnd, parseNot, parseCmp. Two parsers over one language drift unless the second says where it copied from.

Shorthands after the first are immediate tokens, so whitespace decides:

tag_name: (_) => token(/%[a-zA-Z0-9_][a-zA-Z0-9_-]*/),
_class_immediate: (_) => token.immediate(CLASS),

%p on one line and .note on the next are two elements. Only the absence of whitespace between them makes %p.note one.

An unrecognized line is text, which the grammar spells as a token with negative precedence so every other line form wins at the same position.

Queries

queries/highlights.scm is a list of patterns where a later one wins, so general captures come first and specific shapes override them below.

A call gets @function.call by shape:

(call
  function: (identifier) @function.call)

Whether that name is a transform or a helper depends on what the app registered at runtime, so the shape is highlighted and there is no list of names to keep current.

queries/injections.scm hands a filter body to the language that owns it:

((filter
  (filter_name) @_name
  (filter_body) @injection.content)
  (#eq? @_name ":javascript")
  (#set! injection.language "javascript"))

The parser is not committed

grammar.js and scanner.c are source. The C parser the CLI writes from them is generated, so it stays out of the repo, and so does src/grammar.json.

nvim-treesitter runs the generate step at install. hml is not in its registry, so I name it myself in ~/.config/nvim/init.lua:

vim.api.nvim_create_autocmd("User", {
	pattern = "TSUpdate",
	callback = function()
		require("nvim-treesitter.parsers").hml = {
			install_info = {
				url = "https://github.com/croaky/hml",
				branch = "main",
				queries = "queries",
				generate = true,
				generate_from_json = false,
			},
			tier = 0,
		}
	end,
})
if not vim.tbl_contains(ts_installed, "hml") then
	require("nvim-treesitter.install").install({ "hml" })
end

The registration goes in the TSUpdate autocmd, because install and update reload that table and drop anything added to it directly.
generate = true builds against the ABI this Neovim speaks, rather than whatever CLI version last ran. laptop.sh installs tree-sitter-cli so the step has something to run. See cmd / laptop.
generate_from_json = false, because the JSON is generated too, so grammar.js stays the only source.
The URL is the GitHub mirror, so a machine that has never cloned hml still highlights .hml.

No revision is pinned, so nothing goes stale as the grammar moves and the update pass has nothing to compare. It installs once. To pick up a grammar change:

:lua require("nvim-treesitter.install").install({ "hml" }, { force = true })

Testing the grammar

tree-sitter's corpus format is a source block and the tree it should parse to, under test/corpus/:

tree-sitter test

CI runs that after generating the parser:

grammar: tree-sitter generate && tree-sitter test

Generating is itself the check, since a grammar.js the CLI cannot build is a grammar nobody can install.

A corpus example claims to be hml, so a Go test parses every example with the engine. Without it the grammar's examples could drift into a language the engine does not accept.

← All articles