Real-time Telemetry & Ingestion Pipeline
Relevant source files
The following files were used as context for generating this wiki page:
The Real-time Telemetry & Ingestion Pipeline is a critical infrastructure component of the RDM Digital / TAMV platform designed to monitor the operational pulse, health, and events of the urban digital ecosystem. It facilitates the capture of telemetry data from various nodes and services, processing these signals into actionable visualizations such as the "Ecosystem ECG."
The system operates through a decentralized network of nodes (Nexus) that feed data into Supabase Edge Functions. This data is then persisted for auditability and real-time dashboarding, enabling "sovereign digital governance" through continuous observability of the territory's technical health.
Sources: README.md, src/app/pages/CommandCenter.tsx:64-70, src/stores/network.ts:46-55
Architectural Overview
The pipeline follows a serverless architecture utilizing Supabase Edge Functions for ingestion and React-based stores for real-time state management. Data flows from peripheral nodes to a centralized ingestion endpoint, where it is validated and logged.
The diagram above illustrates the end-to-end flow from data generation at the node level to visual representation in the UI.
Sources: supabase/functions/telemetry-ingest/index.ts:18-62, src/components/tamv/tamvcrums-ecg.tsx:10-18, src/lib/tamv/ecgMapper.ts
Data Ingestion & Validation
The primary entry point for telemetry data is the telemetry-ingest edge function. This function enforces strict size constraints and requires authentication for non-public operations.
Ingestion Logic
- Authentication: Validates the Bearer token via the Supabase Auth client.
- Payload Sanitization: Checks if the body is a valid JSON object.
- Size Constraints: Limits the serialized payload to 10,000 characters to prevent resource exhaustion.
- Persistence: Records the event in the
analyticstable with theevent_typeset to the function name.
Sources: supabase/functions/telemetry-ingest/index.ts:7-16, supabase/functions/telemetry-ingest/index.ts:54-61
| Parameter | Type | Description |
|---|---|---|
Authorization | Header | Bearer token for authenticated ingestion requests. |
body | JSON | Telemetry data payload (Max 10KB). |
event_type | string | Automatically assigned based on the function path. |
Sources: supabase/functions/telemetry-ingest/index.ts:57-61
Network Health & Federated Nodes
The system monitors a "Heptafederated Kernel" (TAMV MD-X4) consisting of multiple network layers (L0 through L7). Each layer serves a specific function in the digital territory's infrastructure.
Federated Node Layers
- L0 (Nexus-RDM): The "Nodo Cero" or primary kernel located in Real del Monte.
- L4 (Phoenix Swarm): A P2P mesh network using libp2p.
- L5 (Anubis ZK): A Zero-Knowledge mesh layer for privacy.
- L7 (MSR Bridge): Blockchain bridge to an EVM Sidechain.
Sources: src/stores/network.ts:46-55
Health Metrics
The NetworkStore calculates system health based on:
- Layer Coverage: The percentage of the 8 layers (L0-L7) with active/idle nodes.
- Latency: Average response time across nodes (nominal < 80ms).
- Security Status: Presence of Quantum Encryption and MSR Bridge connectivity.
Sources: src/stores/network.ts:67-78
Ecosystem ECG (Visual Telemetry)
Telemetry data is transformed into a temporal "Electrocardiogram" (ECG) to represent the system's "rhythm." This visualization uses the tamvcrums_logs table, which tracks rhythm, impact scores, and emotional states of the federation.
The sequence diagram shows how raw logs are transformed into visual data points for the dashboard.
Sources: src/components/tamv/tamvcrums-ecg.tsx:10-20, src/lib/tamv/ecgMapper.ts
Telemetry Data Structures
The data model for telemetry events is primarily stored in the analytics and tamvcrums_logs tables.
| Field | Type | Description |
|---|---|---|
ecg_rhythm | integer | Numeric representation of system pulse/rhythm. |
impact_score | numeric | The calculated weight of the event on system health. |
federation_id | text | The federation originating the signal (e.g., 'tecnologica'). |
hash_sha256 | text | Immutable hash of the event payload for auditability. |
Sources: supabase/migrations/202605080001_lovable_federated_kernel.sql:56-62, src/components/tamv/tamvcrums-ecg.tsx:14
Security and Integrity
The pipeline incorporates "Foresic Resilience" and "Immutable Chain of Custody" through several mechanisms:
- Audit Signatures: Every critical resource change is signed and stored in the
audit_signaturestable. - Quantum Encryption: The network store tracks the status of
quantumEncryptionActiveto secure node-to-node communication. - MSR Bridge: Multi-State Records (MSR) act as a ledger for critical events, ensuring data cannot be altered post-ingestion.
Sources: src/stores/network.ts:28, supabase/migrations/202605080001_lovable_federated_kernel.sql:72-78, src/components/rdm/territory.tsx:106-110
Summary
The Real-time Telemetry & Ingestion Pipeline provides the RDM Digital platform with a "sovereign OS" capability to monitor its own territory without dependence on external cloud monitoring suites. By processing federated node signals into the Ecosystem ECG and persisting them through immutable audit tables, the system ensures technical accountability and operational transparency within the TAMV ecosystem.