fix(db): bootstrap Projects for development push databases that predate them

Merged
WA
8d ago
+623-31

Summary

  • Dev schema push (SIM_DEV_DB_PUSH=1 db:push) refused any database that has workspaces but no Project schema, so a push-managed database from before Projects could never sync: push can't backfill workspace.project_id and the guard refuses every unenforced Project state
  • In push mode, reconcile-project-membership.ts --prepare now expands those databases: it replays 0394_project_foundation + 0406_workspace_project_column verbatim (minus 0406's leading migrator batch COMMIT, so the foundation commits atomically with the rollout marker)
  • push.ts then sees the rollout marker on its fence session and runs one Drizzle push with workspace.project_id still nullable (SIM_DB_PUSH_PROJECT_BOOTSTRAP=1, read only by schema.ts for that column), so an older database gains every column 0032 reads (e.g. the fork parent). reconcile-project-membership.ts --bootstrap then runs the registered 0032_project_membership through runScriptMigrations on its single session (recorded exactly as migrate.ts records it), and the ordinary push and reconcilers follow. drizzle.config.ts keeps the connector and rollout marker out of Drizzle's diff until 0032 retires them
  • Runs inside the existing push fence: the fence's transaction-level advisory lock on the migration key keeps migrate.ts and other pushes out; the fence holds no table locks or snapshot, so 0406's concurrent index build and 0032's workspace barrier don't wait on it
  • Resume is narrow: a rollout marker that the drizzle journal doesn't account for (push never creates it, versioned 0406 journals it) with an empty connector. Crash after the expansion, Project conflicts reported by discovery, or a failed enforcement phase all resume on the next push
  • A fresh bootstrap requires no Project table, connector, marker, or project_id column on any public table. Every other state behaves as today: connector with memberships, connector or Projects without the column expansion, a nullable column without Projects, a versioned expansion awaiting 0032, and any push without SIM_DEV_DB_PUSH=1 still refuse with the same message. migrate.ts is untouched
  • drizzle-kit patch: push added PostgreSQL unique constraints after the foreign keys of the same diff, so the composite folder FK on its new (id, project_id, resource_type) unique always failed on an existing table. Unique constraints now come right after new columns. In the noninteractive push policy, the "truncate before adding a unique constraint?" prompt (which aborts without a TTY) is answered with its non-destructive choice; duplicates still fail the push

Type of Change

  • Bug fix

Testing

  • New packages/db/scripts/project-push-bootstrap.integration.ts: pushes the real schema, strips every Project object to build a pre-Project template, then per test runs the real scripts/push.ts --force (fence included). It covers bootstrap + push + replay, a database missing a column 0032 reads (fork parent dropped), (memberships, fork family, archived Project, validated/deferrable constraints, connector and marker retired, 0032 recorded), resume after a discovery conflict, atomic rollback when the workspace barrier is busy, a migration holding the lock, no-op without SIM_DEV_DB_PUSH, and seven refusal states (incl. orphaned folder Project ownership) that must keep schema, workspaces and connector byte-identical
  • Red/green: the bootstrap tests fail on the pre-fix reconciler with the refusal message. Reverting each guard separately (journal check, connector-populated check, batch-commit atomicity, push-mode gate, strict fresh shape, strict resume shape, the nullable first push) turns its named test red
  • New push.integration.ts case for a composite FK on a new unique constraint of a populated table; it goes red with either drizzle hunk removed
  • Also checked locally: databases pushed from the last pre-Project schema.ts and from the last pre-forking schema.ts, seeded with workspaces, then pushed with the current code: both bootstrap and replay. The pre-forking one first hits an unrelated Drizzle diff error dropping a webhook FK that never existed (also on an empty copy, before this change); the rerun resumes and completes
  • scripts/push.integration.ts, project-contract, project-authority, workspace-project-column integration suites (120 tests) pass on PostgreSQL 17; push.test.ts; packages/db type-check; lint; check:utils, check:comment-hygiene, check:test-patterns, check:file-names

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)