Telemetry & Observability Logs
Relevant source files
The following files were used as context for generating this wiki page:
The Telemetry and Observability system in mdeapp provides comprehensive tracking of AI agent operations, tool executions, and search intents. It is designed to facilitate production-safe AI tracing by capturing performance metrics (durations), resource consumption (token usage), and functional outcomes (status, errors) without relying on external observability heavy-weights.
This system spans the application runtime—using a shared RequestContext to accumulate "spans" during a single turn—and the persistence layer, where structured logs are stored in Supabase for analysis. Key components include specialized telemetry builders for AI turns and dedicated database schemas for search logs and grounding failures.
AI Run Telemetry
The project implements a custom telemetry schema (version agt-00c-v1) to audit the behavior of AI agents during a interaction turn. This telemetry is primarily focused on "Turn Telemetry," which aggregates data from multiple tool calls into a single metadata payload.
Telemetry Lifecycle
Telemetry data is collected throughout a "turn" by recording tool spans and token usage deltas. At the end of the turn, this data is summarized and formatted into a TurnTelemetryPayload for storage in the ai_runs table metadata.
Core Data Structures
The system uses several interfaces to manage audit data:
| Interface | Purpose | Key Fields |
|---|---|---|
ToolSpan | Individual tool execution record | tool, ms, status, ts |
TokenUsage | Cumulative token counts | input_tokens, output_tokens, total_tokens |
ToolSpanSummary | Aggregated turn statistics | tool_count, slowest_tool, tool_error_count |
TurnTelemetryPayload | Final metadata object | telemetry_version, model_name, turn_duration_ms |
Sources: src/mastra/lib/tool-audit-context.ts:10-14, src/mastra/lib/tool-audit-context.ts:31-38, src/mastra/lib/mastra-telemetry.ts:8-21
Tool Audit Context
The tool-audit-context module manages the accumulation of data within a RequestContext. It uses specific keys to store state across different functions during a single request.
Audit Keys
mdeaiUserId: Tracks the user initiating the AI turn.mdeaiToolSpans: An array ofToolSpanobjects.mdeaiTokenUsage: A running total of tokens consumed by the model.
Sources: src/mastra/lib/tool-audit-context.ts:3-7
Accumulation Logic
The system provides additive recording functions. recordToolSpan appends new timing data to the context, while recordTokenUsage merges new token counts with existing totals.
Database Observability Schema
The Supabase schema includes dedicated tables for observability, enabling long-term analysis of search performance and AI reliability.
search_logs Table
This table records every search intent processed by the system. It tracks latency, tool selection, and whether advanced techniques like grounding or hybrid search were utilized.
| Field | Type | Description |
|---|---|---|
query_text | text | The raw user query. |
intent | text | The classified user intent. |
slots | jsonb | Extracted search parameters. |
latency_ms | int | Execution time for the search. |
zero_results | boolean | Generated column (results_count = 0). |
Sources: supabase/migrations/20260530120800_data047_search_logs_observability.sql:3-21
Failure and Signal Logs
grounding_failures: Captures instances where the AI could not verify claims against trusted data, including the failure reason and latency.signal_generation_logs: Tracks the background generation of "signals" (e.g., luxury scores, safety scores) for entities, including model versions and confidence scores.
Sources: supabase/migrations/20260530120800_data047_search_logs_observability.sql:34-42, supabase/migrations/20260530120800_data047_search_logs_observability.sql:49-58
Debugging and Utility
Local Debugging
The system supports structured console logging for developers. When the environment variable LOG_LEVEL is set to debug, turn telemetry payloads are emitted as JSON strings prefixed with [mastra-telemetry].
Sources: src/mastra/lib/mastra-telemetry.ts:89-92, src/mastra/lib/mastra-telemetry.test.ts:88-103
Model Name Resolution
A utility function resolveAgentModelName is used to extract the model ID from various Mastra Agent configurations, defaulting to gemini-3.5-flash if the model configuration is missing or dynamic.
Sources: src/mastra/lib/mastra-telemetry.ts:24-38
Summary
The Telemetry & Observability system provides a robust framework for auditing AI behavior in mdeapp. By combining in-memory context tracking during runtime with structured database logging, it enables developers to monitor tool performance, track token costs, and diagnose search failures effectively in a production environment.