Convert analytics read models to bounded microbatches

Merged
AS
6mo ago
+922-354

Summary

  • Adds bounded sleep, activity sensor, and activity location microbatch intermediates, plus compact activity aggregate intermediates.
  • Rewires resting heart rate and activity summary rows to read bounded intermediates and includes them in production DBT_SAFE_MODELS.
  • Adds SQL structure tests and updates analytics docs plus the production incident baseline.

Verification

  • pnpm lint
  • pnpm tsc --noEmit
  • cd packages/server && pnpm tsc --noEmit
  • cd packages/web && pnpm tsc --noEmit
  • bash -lc 'set -a; . ./.env.local; set +a; pnpm test'
  • selected local dbt build for the safe-model set: PASS=10 WARN=0 ERROR=0 SKIP=0

Summary by cubic

Converted analytics read models to bounded microbatches and added compact activity aggregates to make builds reliable and fast. Re-enables scheduled resting heart rate and activity summaries in production.

  • New Features

    • Added microbatch intermediates: sleep_heart_rate_sample, activity_sensor_sample, activity_location_sample (daily batches, short lookbacks).
    • Added aggregate intermediates: activity_sensor_summary_rows, activity_location_summary_rows (location picks the best provider per activity).
    • Expanded DBT_SAFE_MODELS to include the above plus resting_heart_rate_sleep_window and activity_summary_rows.

  • Refactors

    • activity_summary_rows now joins the aggregate intermediates instead of raw streams; preserves elevation defaults and uses max_threads=1 for offline builds.
    • resting_heart_rate_sleep_window now aggregates from sleep_heart_rate_sample and limits threads.
    • Added SQL structure tests to lock safe-model order and dependency boundaries; updated analytics docs, analytics sources (v_activity, v_activity_members), and the production incident baseline.

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Fixed Resting Heart Rate chart displaying stale data; metrics now refresh more consistently with underlying health data updates.

  • Documentation

    • Updated analytics architecture documentation to reflect improved data processing strategy.

  • Tests

    • Added validation tests for analytics data model dependencies and build order.

Review Change Stack