postgres / safe migrations
These DDL patterns avoid blocking queries during migrations. I enforce them with a migration linter in Go.
Adding indexes
Bad (blocks writes):
CREATE INDEX index_foo ON bar (col)
Good (run outside transaction):
CREATE INDEX CONCURRENTLY index_foo ON bar (col)
Adding foreign keys
Bad (blocks writes on both tables):
ALTER TABLE foo ADD COLUMN bar_id bigint REFERENCES bar(id)
Good (two migrations):
-- Migration 1: add FK without validating
ALTER TABLE foo
ADD COLUMN bar_id bigint,
ADD CONSTRAINT foo_bar_id_fkey
FOREIGN KEY (bar_id) REFERENCES bar(id) NOT VALID;
CREATE INDEX CONCURRENTLY index_foo_on_bar_id ON foo (bar_id);
-- Migration 2: validate
ALTER TABLE foo VALIDATE CONSTRAINT foo_bar_id_fkey;
I name a foreign key <table>_<column>_fkey, the name Postgres gives
a key it names itself. An error that names the key then says which
table and column failed. Rails named keys with a hash, such as
fk_rails_<hash>, and a reader had to search the schema to read the
error. One migration renamed 90 of those:
ALTER TABLE foo
RENAME CONSTRAINT fk_rails_0123456789 TO foo_bar_id_fkey;
A rename changes only the catalog, but it holds an ACCESS EXCLUSIVE
lock on each table until the transaction commits. The runner's 10s
lock_timeout limits each wait. A long job that holds a table can
fail the deploy, and the next deploy runs the migration again.
The rename found one column with two foreign keys to the same table.
One had ON DELETE CASCADE and one had no action. With two delete
actions on one column, the order of their triggers decides what a
delete does. I dropped the key with no action.
Setting NOT NULL
Bad (blocks reads/writes while checking every row):
ALTER TABLE foo ALTER COLUMN bar SET NOT NULL
Good (two migrations):
-- Migration 1: add check constraint without validating
ALTER TABLE foo
ADD CONSTRAINT foo_bar_not_null CHECK (bar IS NOT NULL) NOT VALID;
-- Migration 2: validate, set NOT NULL, drop redundant constraint
ALTER TABLE foo VALIDATE CONSTRAINT foo_bar_not_null;
ALTER TABLE foo ALTER COLUMN bar SET NOT NULL;
ALTER TABLE foo DROP CONSTRAINT foo_bar_not_null;
Backfilling data
Bad (locks table for duration):
BEGIN;
ALTER TABLE foo ADD COLUMN bar text;
UPDATE foo SET bar = 'default';
COMMIT;
Good (separate migrations, batched updates):
-- Migration 1: add column
ALTER TABLE foo ADD COLUMN bar text;
-- Migration 2: backfill in batches
UPDATE foo SET bar = 'default'
WHERE id IN (SELECT id FROM foo WHERE bar IS NULL LIMIT 10000);
-- repeat until done
Adding check constraints
Bad (blocks reads/writes):
ALTER TABLE foo ADD CONSTRAINT bar_positive CHECK (bar > 0)
Good (two migrations):
-- Migration 1
ALTER TABLE foo ADD CONSTRAINT bar_positive CHECK (bar > 0) NOT VALID;
-- Migration 2
ALTER TABLE foo VALIDATE CONSTRAINT bar_positive;
Removing columns
Bad (errors if app code still references column):
ALTER TABLE foo DROP COLUMN bar
Good: deploy application code without the column references, then drop the column.
Editing a migration after it ran
The runner records each version it applies. A later edit to that file
does nothing on a database that ran it. One change merged a migration,
every dev database ran it, and a second change edited the same file.
The databases kept the old function, and the tree said the new one.
The runner dumps db/schema.sql from the database, so the next three
changes each carried a diff that reverted the edit.
Now the runner stores each file's checksum when it applies the file:
ALTER TABLE schema_migrations ADD COLUMN checksum text;
Before a run, it compares each recorded checksum with the file on
disk. A mismatch stops the run. The message names the file and the two
fixes: restore the file, or roll back past it and migrate again. A row
recorded before the column has a NULL checksum. The runner reads it
as unknown and reports nothing it cannot prove.
The runner also writes db/schema.sql only when the run changed the
database. When nothing changed and the dump disagrees with the file,
the database is wrong, so the runner reports it and leaves the file.