feat(dashboards): Omi TV device-vs-organic funnel row and gateway errors by lane

Merged
GI
2d ago
+831-4

What

Omi TV v3: a funnel row below the fixed wall (scroll down; the 27-row TV wall itself is unchanged). The row splits new users into "device" (paired Omi hardware within 48 hours of first seen; Device Paired with device_vendor omi, or type omi/openglass on older mobile builds) and organic. Pairing only another vendor (Apple Watch, Limitless, ...) counts as organic. David ruled 11-02 that the funnel shows as tiles only, with "paired" standing in for device owners.

  • Funnel · device vs organic: the latest matured weekly new-user cohort, step by step: share of new users, onboarded (iOS, Android and macOS entrants only; Windows and web never send Onboarding Completed), conversation on day 1, conversation in week 1, activated (2+ active days in week 1). Kept (activated, then active in days 7–35) comes from the cohort four weeks older, the newest one old enough to measure it.
  • Activated · device vs organic: week-one activation for six weekly cohorts.
  • Funnel by platform: week-one activation by entry platform, with the cohort size in each label.
  • Install and first summary are not shown. PostHog sees nobody before sign-up, and first summary has no event. The panel descriptions say so.

Core Features: gateway errors by lane (David ruled "add it" on 11-02). These are two panels in the AI responses row:

  • Failed requests per second by lane (top 10).
  • Error rate by lane, for lanes above ~1 request a minute (top 10).

Both read the lane_outcome:llm_gateway_requests:rate5m_offset5m recording rule that is already on main (#24167/#24184). Thresholds stay unchanged (green 99%+, red under 95%), and Core Features stays a separate dashboard from Omi TV.

Data source and cost

The funnel is one new PostHog leg, deviceFunnel, inside the existing activation series. It runs one HogQL read per weekly cohort, six in total, through the existing periods() partial-retry and backoff path. A cohort-filtered read took about 3 s live.

The scan runs 28 days past each period's end, so the days 7–35 Kept window is complete for the older cohort. Stopping the scan at the period end would have cut that window to under a week.

Live check (read-only, PostHog project 302298, 2026-11-02)

CohortUsersDevice shareActivated, deviceActivated, organicKept, deviceKept, organic
Latest matured week (end 11-02)1,59320.8%65.0%12.0%––
Four weeks older (end 10-05), before the vendor filter1,11921.4%75.0%19.8%52.9%11.5%

The organic denominator includes people with no client platform (unknown: 346 in the latest cohort, with no onboarding or activity) and web sign-ups. This matches the wall's existing activation denominator (the same cohort as captureSourceQuery). The platform chart leaves unknown out.

Activated can read slightly above "conversation, week 1", because activation counts foreground days as well as captures.

Tests

  • vitest run lib/__tests__/wall-series.test.ts: 61 passed, including the 5 new funnelFrames tests.
  • python3 -m unittest discover web/admin/grafana: 16 passed. These cover the builder-matches-artifact check and the new funnel-row geometry check.
  • backend/test.sh with the Core Features contract set (test_core_feature_headline_contract, test_daily_summary_monitoring_contract, test_monitoring_dashboard_query_budget, test_monitoring_notification_contract): 29 passed.
  • make preflight: 20 checks passed.

Deploy

  • Omi TV: the web/admin deploy (the funnel leg and frames) plus Grafana board provisioning for omi-tv-v2.json.
  • Core Features: Grafana dashboard provisioning (grafana_dashboard_provisioning.yml) for omi-core-features.json.

No backend service deploy is needed.

Product invariants affected

none

🤖 Generated with Claude Code