fix(knowledge): make indexing and connector recovery durable
Merged
IC
1mo ago+29238+29238+01234567890123456789012345678901234567890123456789-535-535-012345678901234567890123456789
Summary
- Save workspace imports as immutable KB-owned files before queueing, using the shared upload attachment and orphan-cleanup guard.
- Recover abandoned stored connector documents independently of source sync. Atomically install a new processing generation and outbox delivery with bounded attempts, runtime, and admission backoff.
- Treat shared provider capacity as a durable deferral. Preserve content and member progress across GitHub waits, pin immutable tree/blob reads, and reconcile removals only after a complete listing.
- Preserve safe, structured source failures and retry only positively identified transient database reads. Set HTTP body framing in the shared pinned transport.
- Keep regular KB and organization Search ownership, permissions, complete-index publication, and billing behavior intact. Apply the additive migration before deploying the workers.
Type of Change
- Bug fix
Testing
- 4,080 unit tests passed across affected knowledge, connector, processing, storage, deadline, and provider-capacity paths.
- 866 shared HTTP security tests passed, including real-socket body framing and redirect regressions.
- 328 integration tests and 13 PostgreSQL access/search tests passed in disposable PostgreSQL/Redis, including synthetic content against live OpenAI/Mistral APIs. The 42 skipped tests require separate opt-in provider or scale fixtures.
- Type check, lint, all 46 CI audits, API validation, docs manifest, block registry, and migration safety checks pass.
- Recovery coverage includes concurrent admission, duplicate/stale delivery, atomic rollback, blocked owners, organization Search authorization, immutable imports after source deletion, and concurrent-index migration replay.
Checklist
- Code follows project style guidelines
- Self-reviewed my changes
- Tests added/updated and passing
- No new warnings introduced
- I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)