feat(usage): add enterprise organization usage monitoring

Merged
IC
2mo ago
+26110-722

Enterprise org admins have no way to see where their pooled credits go. The billing page hides the usage-limit field and the credit-usage drill-down for enterprise, and credit-usage-view is personal-scope only — so the one audience that negotiates a pooled commitment is the one audience that can't watch it burn.

Everything needed already lived in usage_log, the single source of cost truth, where every row billed to an org is stamped billing_entity_type='organization' at charge time. This is the read surface over it, plus one narrow write so BYOK usage is captured instead of discarded.

Usage tracking

Settings → Organization → Usage tracking, gated on hosted + enterprise with a USAGE_MONITORING_ENABLED self-hosted override, matching every other enterprise feature.

Five tabs, one question each: Overview (how much, and what kind of work), Members, Workspaces (select one to drill into its Sources and Workflows), Models, BYOK. Behind an All events action sits the full ledger, mirroring how settings/billing/credit-usage already relates to Billing.

Design notes worth reviewing:

  • Only the visible tab's dimension is fetched. Three of six breakdown dimensions aren't index-covered and heap-fetch per row, so a tab nobody opens never pays for one. Custom ranges are capped at 92 days for the same reason.
  • Every ranked list closes with an explicit Other (N more) row, so each list reconciles to the headline figure. Five lists that don't add up is the classic "the numbers are wrong" bug.
  • One window abstraction. buildUsageAnalyticsScope carries the reporting branch copied from getBillingPeriodUsageCost — reporting periods match created_at, stripe/default periods match the billing_period_* stamps exactly. Diverging here is how this panel would come to disagree with the billing page about the same period.
  • Buckets are date_trunc, not the logs dashboard's epoch-modulo arithmetic. A billing period starts at an arbitrary instant, so modulo buckets would cut every day mid-afternoon and each bar would straddle two calendar days.
  • readUsageTimeSeries groups by the output alias, not the expression — Postgres matches GROUP BY syntactically, and re-rendering the fragment produces a textually different one that it rejects outright.

Track unbilled (BYOK) model usage

BYOK spans already reached costSummary with real token counts and were dropped at exactly one gate, if (modelData.total > 0). A new model_unbilled category records them at zero cost.

  • Written only at the terminal execution boundary, keyed on stableEventKey with the existing onConflictDoNothing, so a resumed run writes once with cumulative tokens and the cost reconciliation path is untouched.
  • Written only when billing attribution is already resolved, and the targets.length === 0 bail stays keyed on billable targets — otherwise a BYOK-only run with no billable target (a zero-base-charge custom-block child) would newly hit the attribution requirement and start erroring.
  • Safe by construction: every billing read over usage_log is SUM(cost), so zero-cost rows change no total anywhere. A distinct category (rather than cost = 0 on category='model') means existing WHERE category = 'model' queries cannot see them unless they ask.

Rollout: the enum value must be released before any code writes it. Migration 0310 is ADD VALUE IF NOT EXISTS, and Postgres cannot use a new enum value in the transaction that adds it.

Collapse organization settings onto the workspace plane

/organization/[id]/settings/* was a second plane with no UI entry point anywhere — nothing linked to it from a sidebar, menu, or button — whose nine sections all already render on the workspace plane (navigation.test.ts already asserted that parity). Deleted, along with its renderer and unavailable page.

planes.organization is replaced by a single unified.organizationSection marker that now derives both ORGANIZATION_PLANE_UNIFIED_SECTIONS and the section map in the workspace gate — the latter was a hand-written literal that had to stay in lockstep with the registry.

Two things that would actually have broken, fixed:

  • Usage-threshold emails built /organization/{id}/settings/billing as their CTA. Now workspace-scoped.
  • POST /api/v1/admin/organizations and instance-org provisioning could strand an admin with zero workspaces. Both now backfill one, the same way enterprise-owner-claim already did.

Plain org members keep their read-only roster access via allowNonOrgAdmin plus a members-only carve-out in the gate, rather than losing a capability silently.

Shared chart module

LineChart moves to components/charts/ with a matching BarChart sibling built from the same geometry, tooltip, and theme modules, so the two cannot drift apart. The move also breaks the import edge from the chart to logs/utils → @/blocks/registry, which would otherwise have pulled the entire executable block registry into the settings chunk.

The logs dashboard renders identically — the lift held a zero-visual-diff bar, including keeping two pre-existing quirks verbatim (see below).

Permissions

Four layers agree and are now tested: sidebar visibility, the section page gate, the settings/usage/events sub-route (which sits outside [section] and inherits none of its checks, so it repeats the gate itself), and the API. The operation names session only — an org's pooled ledger discloses every member's model spend, so an API key is refused before authority is even checked. Authority is checked before entitlement, so a non-admin learns nothing about the org's plan.

Per-member credit caps stay hosted-only. The usage-limit route 404s where Sim doesn't own billing and checkOrganizationMemberUsageLimit no-ops, so the Manage credits row action is gated on isHosted rather than offering an action that could only fail.

Testing

  • bunx turbo run type-check — 26/26
  • bun run test — 2,377 files, 35,091 tests, 0 failures
  • check:api-validation:strict, check:react-query, check:client-boundary, check:native-typecheck — all pass
  • All 12 analytics reads verified against a real local Postgres for reconciliation (sum(rows) + other === total, sum(series) === totals), since the global drizzle mock makes unit tests SQL-blind here
  • Query plans checked against the prod read-only replica (4.8M rows). An earlier COUNT(DISTINCT user_id) cost 830ms of a 909ms summary for a figure that was never rendered; removing it took the summary to 74ms.

Notes for review

Two pre-existing bugs found and deliberately not fixed, because both would change the logs dashboard and this PR held a zero-visual-diff bar on that lift:

  1. stroke='hsl(var(--border))' on the chart axis lines — --border is a hex (#d8d8d8), never an HSL triplet, so this renders as hsl(#d8d8d8), which is invalid, and the axis lines have never painted in either theme. Verbatim from the original line-chart.tsx.
  2. The grid lines read --muted from the block globals.css marks @depricated … Do not modify. The new BarChart track uses the live --border token instead — light-mode --muted is #f5f5f5, which at 12% opacity on white is a 1.2/255 delta, i.e. invisible.

Screenshots in the docs page are Cloud; the BYOK tab has no image yet because model_unbilled only starts accumulating once this ships.

🤖 Generated with Claude Code