Service Level Objectives (SLOs)
Relevant source files
The following files were used as context for generating this wiki page:
Service Level Objectives (SLOs) define target reliability levels for the Financial Asset Relationship Database. This system tracks performance metrics and error budgets to ensure enterprise-grade stability. The observability layer monitors API health, graph rebuild operations, and data consistency.
Sources: monitoring/README.md:1-5, AUDIT_REPORT.md:522-525
Observability Architecture
The monitoring infrastructure integrates Prometheus metrics, Loki log aggregations, and Grafana dashboards. The system relies on the Phase 3 Observability Layer for event tracking and metric collection.
Data Flow for Monitoring
The following diagram illustrates how metrics propagate from the application to the alerting system:
The flow shows metrics and logs feeding into aggregation servers for alerting and visualization.
Sources: monitoring/README.md:38-44
Multi-Window Burn Rate Alerting
The system implements the Google SRE standard for Multi-Window Multi-Burn-Rate Alerting. This method identifies budget exhaustion risks while minimizing false positives. The alerts use short windows to detect sudden spikes and long windows to verify persistence.
Sources: monitoring/README.md:27-29
Alert Thresholds
The alerting logic categorizes budget consumption into two primary severities:
| Alert Type | Threshold | Severity | Description |
|---|---|---|---|
| Fast Burn | 24 Hours | Critical | Monthly budget exhausts within 24 hours. |
| Slow Burn | 3 Days | Warning | Monthly budget exhausts within 3 days. |
Sources: monitoring/README.md:31-35
SLO Evaluation and Compliance
The slo_evaluator.py component calculates compliance metrics against defined targets. It processes raw metrics to determine the current status of the error budget.
Monitored Categories
The system prioritizes monitoring for the following areas:
- Global API Health: Tracks latency and error rates for all REST endpoints.
- Rebuild Operations: Monitors the duration and failure rates of the graph rebuild lifecycle.
- Log-to-Metric Aggregations: Uses
loki-recording.ymlto compute metrics from log counts.
Sources: monitoring/README.md:6-14, AGENTS.md:145-152
Dashboard Configuration
The slo-compliance.json file provides the configuration for Grafana dashboards. These dashboards visualize burn rates, remaining error budgets, and historical compliance trends.
You must configure the following data sources in Grafana to enable these visualizations:
- Map the Prometheus data source in
provisioning/datasources.yml. - Set the dashboard provider to point to the
monitoring/dashboards/directory.
Sources: monitoring/README.md:18-25
Conclusion
The SLO framework ensures the project meets its Enterprise Readiness Roadmap goals. By utilizing multi-window burn rate alerts and standardized Prometheus metrics, the system provides deterministic health reporting and disaster recovery signals.
Sources: AUDIT_REPORT.md:518-522, monitoring/README.md:38-40