Stats Dashboard

Relevant source files

The following files were used as context for generating this wiki page:

• packages/stats/src/server.ts
• packages/coding-agent/src/cli/stats-cli.ts
• packages/coding-agent/src/export/html/template.js
• packages/typescript-edit-benchmark/src/report.ts
• packages/coding-agent/src/autoresearch/tools/log-experiment.ts

The Stats Dashboard is a comprehensive monitoring and visualization system within the Oh My Pi (OMP) project designed to track AI usage, costs, and performance metrics. It provides developers with a high-level overview of token consumption, model performance, error rates, and financial expenditures across different projects and sessions.

The system is accessible via a command-line interface (CLI) that triggers a local web server, serving a Single Page Application (SPA). This dashboard aggregates data from session files, providing both real-time synchronization and historical analysis of AI interactions.

Sources: packages/coding-agent/src/cli/stats-cli.ts:184-210, packages/stats/src/server.ts:251-300

Architecture and Components

The Stats Dashboard follows a client-server architecture. The server is responsible for data aggregation from the local database and session files, while the client provides a React-based interactive interface.

Dashboard Architecture Flow

The following diagram illustrates the data flow from raw session files to the user interface.

This diagram shows how the system synchronizes local session data into a structured database to serve the frontend.
Sources: packages/coding-agent/src/cli/stats-cli.ts:110-120, packages/stats/src/server.ts:211-249

Server Component

The server is built using Bun and provides a REST API to expose statistics. It handles the extraction of embedded client assets if the binary is compiled, or builds the client from source during development.

• Initialization: The server ensures the client build exists and determines if it should use an embedded archive or static directory.
• API Routes: Provides endpoints for overview stats, model-specific data, cost breakdowns, and recent request details.
• Synchronization: Coordinates the syncAllSessions process which parses local session logs into the database.

Sources: packages/stats/src/server.ts:15-115, packages/stats/src/server.ts:211-249

CLI Integration

The system is invoked via the omp stats command. The CLI handles argument parsing (port, JSON output, summary mode) and manages the lifecycle of the dashboard process.

export async function runStatsCommand(cmd: StatsCommandArgs): Promise<void> {
    const { getDashboardStats, syncAllSessions, getTotalMessageCount, startServer, closeDb } = await import(
        "@oh-my-pi/omp-stats"
    );

    // Sync session files first
    const { processed, files } = await syncAllSessions({ onProgress: progress.onProgress });
    
    // Start the dashboard server
    const { port } = await startServer(cmd.port);
    console.log(chalk.green(`Dashboard available at: http://localhost:${port}`));
}

Sources: packages/coding-agent/src/cli/stats-cli.ts:107-130

API Endpoints

The dashboard server exposes several RESTful endpoints to provide data to the frontend.

Endpoint	Method	Description
/api/stats	GET	Returns the primary dashboard statistics (overall, byModel, byFolder).
/api/stats/overview	GET	High-level summary of usage metrics.
/api/stats/costs	GET	Breakdown of token costs (input, output, cache).
/api/stats/recent	GET	List of the most recent AI requests.
/api/request/:id	GET	Detailed data for a specific request ID.
/api/sync	POST	Triggers a fresh synchronization of session files.
/api/stats/gain	GET	Performance gain metrics for specific projects.

Sources: packages/stats/src/server.ts:211-249

Data Metrics and Visualization

The dashboard tracks a variety of metrics to provide a granular view of AI agent behavior.

Usage Statistics

The overall statistics include counts for total requests, failed requests, and token usage categorized by input, output, and cache. It also calculates error rates and cache hit rates.

• Premium Requests: Tracks high-cost model usage.
• Latency Metrics: Records Average Duration and Time to First Token (TTFT).
• Throughput: Calculates Tokens per second (Tokens/s).

Sources: packages/coding-agent/src/cli/stats-cli.ts:143-162

Session Tree Visualization

For individual sessions, the system uses a tree structure to represent the conversation flow, including branching and tool executions.

The visualization logic (implemented in template.js) allows for deep linking into specific messages using leafId and targetId.
Sources: packages/coding-agent/src/export/html/template.js:70-130, packages/coding-agent/src/export/html/template.js:585-615

Specialized Dashboards

Experiment Logging (Autoresearch)

The log_experiment tool integrates with the dashboard to visualize scientific research runs. It records primary metrics, status (keep/discard/crash), and "ASI" (free-form structured metadata).

• Confidence Scores: Measures the noise floor of experiments.
• Scope Deviations: Flags when an agent modifies files outside the permitted research scope.

Sources: packages/coding-agent/src/autoresearch/tools/log-experiment.ts:153-195

Edit Benchmark Reporting

The system generates Markdown reports for benchmarking code editing capabilities. These reports summarize:

• Task Success Rate: Percentage of successfully completed edits.
• Tool Usage Rate: Frequency of read, edit, and write tool calls.
• Edit Failure Categories: Classified reasons for edit failures (e.g., syntax errors, context mismatch).

Sources: packages/typescript-edit-benchmark/src/report.ts:100-150

Conclusion

The Stats Dashboard serves as the central observability hub for Oh My Pi. By aggregating disparate session logs into a unified relational database and providing a web-based interface, it enables developers to optimize AI performance, manage costs, and debug complex agentic workflows with high precision.