fix(db): reclaim Capgo-EU swap pressure without upgrading

Merged
RI
3mo ago
+1294-4

Summary (AI generated)

  • Add migration to cut primary DB swap pressure without a compute upgrade: batched cleanup_queue_messages (2-day retention, hourly), hourly net._http_response truncate, batched 30-day audit cleanup, slim app_versions audit payloads, and hourly nulling of leftover dual-storage manifest arrays.
  • Allow nulling migrated manifest / native_packages after upload in check_encrypted_bundle_on_insert so reclaim is no longer blocked by bundle_already_ready.
  • Add ops reclaim/verify SQL scripts and DB tests for the new cleanup paths.

Motivation (AI generated)

Capgo-EU was swap-thrashing and restarting. Primary traffic is small (devices/stats nearly empty); pressure came from unreclaimed bloat (~5GB empty net._http_response, multi-GB stale pgmq archives, fat audit TOAST, and dual-storage app_versions.manifest arrays that could not be cleared).

Business Impact (AI generated)

Keeps the current DB size workable, reduces OOM/restart risk for console and internal jobs, and avoids an unnecessary primary compute upgrade.

Test Plan (AI generated)

  • bun run supabase:db:reset applies 20260722082019_fix_supabase_swap_memory.sql
  • bun run supabase:with-env -- bunx vitest run tests/cleanup_swap_memory.test.ts tests/enforce-encrypted-bundles.test.ts (26 passed)
  • Deploy migration to Capgo-EU
  • Run scripts/ops/reclaim_supabase_swap.sql in a maintenance window
  • Confirm with scripts/ops/verify_supabase_swap.sql (sizes down, archives <2d, cron tasks present, fewer server restarted rows)

Generated with AI

Made with Cursor

Summary by CodeRabbit

  • New Features
    • Added automated, scheduled maintenance to reclaim stale queue data, truncate stored HTTP responses, delete old audit logs, and null fully migrated version manifests.
    • Reduced queue/audit payload bloat by omitting large fields when generating queued jobs and audit entries.
    • Strengthened encrypted bundle validation on updates, with support for the dual-storage reclaim pathway.
  • Maintenance
    • Added Postgres verification and manual reclaim/cleanup scripts for the swap pressure scenario.
  • Tests
    • Expanded test coverage for cleanup behavior, audit payload filtering, migration safety, and encrypted bundle handling.