Event Catching & Alerting System
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Event Catching & Alerting System in ManageIQ is a sophisticated framework designed to discover, track, and respond to operational changes across hybrid IT environments. It captures "Management Events" from External Management Systems (EMS), physical hosts, and virtual machines to provide real-time insight into infrastructure status. The system integrates these events with a policy-based alerting engine that can trigger automated responses, notifications, or manual interventions based on predefined conditions.
The core of the system revolves around EventStream and its specialized subclass EmsEvent, which serve as the primary data structures for captured activity. These events are processed through the MiqEventHandler and evaluated against MiqAlert definitions. This ensures that critical state changes—such as VM power transitions, hardware failures, or performance thresholds—are not only recorded for historical reporting but also acted upon immediately to maintain compliance and operational stability.
Sources: app/models/ems_event.rb:11-15, app/models/miq_alert.rb:102-110, README.md:27-31
Event Lifecycle and Architecture
The lifecycle of an event begins with its ingestion from a provider (EMS). Events are captured and placed into a messaging queue (Artemis, Kafka, or internal MiqQueue) before being persisted in the database.
Event Ingestion and Processing Flow
Events are added to the system via the add or add_queue methods. During ingestion, the system performs "object linking," where it attempts to associate the raw event data with existing ManageIQ objects such as VMs, Hosts, Clusters, or Physical Storage.
The diagram shows the transition from raw external provider events through the messaging tier into the core EmsEvent processing logic.
Sources: app/models/ems_event.rb:111-133, spec/models/ems_event_spec.rb:133-210
Management Event Groups
Events are categorized into logical groups and assigned severity levels (e.g., critical, warning). This grouping is used for timeline visualization and filtering.
| Group Name | Description | Example Event Types |
|---|---|---|
| Addition | Creation/Addition of resources | CloneTaskEvent, VM_Created |
| Power | Power state transitions | VmPoweredOffEvent, VmPoweredOnEvent |
| Snapshot | Snapshot operations | AlarmCreatedEvent, AlarmRemovedEvent |
| Security | Security-related changes | LoginEvent, SecurityViolationEvent |
Sources: app/models/ems_event.rb:25-33, spec/models/ems_event_spec.rb:342-370
Alerting Engine
The MiqAlert model defines the logic for evaluating whether a specific condition warrants a notification or action. Alerts can be based on expressions, performance thresholds, or specific event occurrences.
Evaluation Logic
Alerts are evaluated against a "Target" (e.g., a VM or Host). The system uses a postpone_evaluation? check to prevent alert fatigue by honoring a delay_next_evaluation frequency.
This sequence illustrates the asynchronous nature of alert evaluation, utilizing the queue to process complex expressions without blocking the event stream.
Sources: app/models/miq_alert.rb:147-190, app/models/miq_alert.rb:213-228
Notification and Action Types
When an alert evaluates to true, the system can trigger multiple notification types defined in the alert's options.
- Email: Sends an email to specified recipients.
- SNMP: Sends an SNMP trap.
- Automate: Raises an event in the ManageIQ Automation engine (
MiqAeEvent). - EVM Event: Generates an internal event for further policy processing.
Sources: app/models/miq_alert.rb:269-315
Data Models and Relationships
EmsEvent Model
The EmsEvent record stores comprehensive metadata about a management event, including links to source and destination objects.
| Field | Type | Description |
|---|---|---|
event_type | string | The specific type of event (e.g., VmPoweredOffEvent). |
timestamp | datetime | When the event occurred on the provider. |
ems_id | integer | Foreign key to the ExtManagementSystem. |
vm_or_template_id | integer | Linked VM or Template. |
host_id | integer | Linked Host. |
full_data | hash | Serialized blob containing provider-specific event details. |
Sources: app/models/ems_event.rb:111-133, app/models/ems_event.rb:233-242
Object Linking Logic
The system automatically links events to infrastructure objects using provider-specific references (ems_ref). For example, process_vm_in_event! looks for a VM matching the vm_ems_ref or vm_uid_ems provided in the raw event hash.
Sources: app/models/ems_event.rb:148-175, spec/models/ems_event_spec.rb:18-65
Timeline Reporting
ManageIQ uses the Timeline formatter to visualize captured events over time. This system aggregates EventStream data into buckets based on categories (e.g., Power Activity) and renders them for Simile Timelines.
Timeline Formatting Flow
- Data Selection: The report engine queries
EventStreamorPolicyEventtables. - Bucketing: Events are mapped into categories based on
include_set(string matching) orregexes(pattern matching). - Message Generation: The
TimelineMessageclass generates HTML-formatted summaries, including links to affected VMs, Hosts, or Clusters.
Sources: lib/manageiq/reporting/formatter/timeline.rb:21-48, lib/manageiq/reporting/formatter/timeline_message.rb:21-50
Conclusion/Summary
The Event Catching & Alerting System provides the foundational "Control" and "Automate" capabilities of the platform. By normalizing disparate event streams from various cloud and infrastructure providers into a unified EmsEvent model, the system enables consistent policy application across the entire environment. The integration of the alerting engine ensures that administrators are proactively notified of critical changes, while the automation hooks allow the system to self-heal or reconfigure infrastructure in response to real-time events.