Unified Alerting System
Relevant source files
The following files were used as context for generating this wiki page:
- public/app/features/alerting/unified/AGENTS.md
- public/app/features/alerting/unified/components/receivers/TemplateDataExamples.ts
- public/app/features/alerting/unified/utils/template-constants.ts
- pkg/services/ngalert/api/test-data/receiver-exports/unredacted/all-integrations.hcl
- pkg/services/ngalert/api/tooling/cmd/clean-swagger/main.go
- contribute/style-guides/frontend.md
The Unified Alerting System is a centralized framework within Grafana for managing alert rules, contact points, and notification policies. It is designed to handle both Grafana-managed rules and external Alertmanager configurations, providing a consistent user experience for alerting across different data sources. The system integrates frontend components built with React and Redux Toolkit with a backend API (often referred to as ngalert) that handles rule evaluation, state management, and notification routing.
Sources: public/app/features/alerting/unified/AGENTS.md, pkg/services/ngalert/api/tooling/cmd/clean-swagger/main.go:49
System Architecture and Components
The architecture follows a modular pattern where the frontend communicates with the backend via specialized API slices. The frontend is organized into specific domains such as rule management, contact points, and silences.
Logic Flow for Alert Processing
The following diagram illustrates the high-level flow from rule evaluation to notification delivery within the Unified Alerting System.
This flow represents how rules transition from evaluation to notification through policies and contact points.
Sources: public/app/features/alerting/unified/AGENTS.md, pkg/services/ngalert/api/test-data/receiver-exports/unredacted/all-integrations.hcl
Key Frontend Modules
The frontend is structured into several functional directories that manage different aspects of the alerting lifecycle:
| Directory | Responsibility |
|---|---|
rule-editor/ | Handles alert rule creation and editing forms. |
rule-list/ | Provides views for listing and filtering alert rules. |
contact-points/ | Manages the configuration of notification destinations (receivers). |
notification-policies/ | Defines routing logic for alerts based on labels. |
silences/ | Manages suppression of alerts for specific time windows. |
Sources: public/app/features/alerting/unified/AGENTS.md:85-100
Data Fetching and State Management
The Unified Alerting System prioritizes RTK Query for data fetching over legacy Redux patterns. This allows for automatic caching, polling, and standardized state handling across the alerting UI.
API Integration Pattern
The system uses auto-generated API clients where possible, but allows for manual enhancement when specific overrides are required.
The sequence above shows the standard data retrieval pattern using RTK Query hooks.
Sources: public/app/features/alerting/unified/AGENTS.md:105-125, pkg/services/ngalert/api/tooling/cmd/clean-swagger/main.go:53
Core API Slices
- alertRuleApi: Manages alert rule CRUD operations.
- alertmanagerApi: Handles Alertmanager-specific configurations and silences.
- receiversApi: Manages contact points and integration testing.
- prometheusApi: Interfaces with Prometheus-compatible backends for rule evaluation.
Sources: public/app/features/alerting/unified/AGENTS.md:112-115
Notification Templates and Receivers
Notifications are customized using a templating system that allows users to define how alert data is formatted for different contact points.
Default Templates
The system includes standard templates for notification titles and messages, often utilizing the default.title and default.message definitions.
// Example of a default title template structure
{{ define "default.title" }}
[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }}
{{ end }}
Sources: public/app/features/alerting/unified/utils/template-constants.ts:1-18, public/app/features/alerting/unified/components/receivers/TemplateDataExamples.ts:8-15
Supported Integrations
The system supports a wide array of receivers, each with specific configuration parameters.
| Receiver Type | Key Configuration Fields |
|---|---|
| Slack | recipient, token, icon_url, mention_users |
| PagerDuty | integration_key, severity, class, component |
| Webhook | url, http_method, basic_auth_user, hmac_config |
| Opsgenie | api_key, responders, auto_close |
addresses, subject, single_email |
Sources: pkg/services/ngalert/api/test-data/receiver-exports/unredacted/all-integrations.hcl:1-200
Alert Rule Logic
Alert rules can be categorized into two primary types: Grafana-managed (Ruler) and Data Source-managed (External Prometheus/Loki).
- Grafana-managed Rules: Evaluated by the Grafana backend. They support multiple data sources and complex query structures.
- External Rules: Managed and evaluated by external services like Prometheus or Cortex, but visible and sometimes manageable within the Unified Alerting UI.
Rule Relationships
Permissions and Access Control
The system utilizes Role-Based Access Control (RBAC) to manage user abilities within the alerting domain. Common actions include AlertingRuleRead, AlertingRuleCreate, and AlertingRuleUpdate. The useAbilities hook is the primary interface for checking these permissions in the frontend.
Sources: public/app/features/alerting/unified/AGENTS.md:290-300, contribute/style-guides/frontend.md
Summary
The Unified Alerting System provides a robust, scalable framework for alert management. By unifying disparate alerting mechanisms into a single interface and API, it allows for sophisticated notification routing, flexible templating, and consistent governance across an entire observability stack. Key technical pillars include RTK Query for efficient state management and a wide array of receiver integrations for flexible incident response.