Webhooks & Event Listeners
Relevant source files
The following files were used as context for generating this wiki page:
The eval-platform uses a decoupled architecture to manage asynchronous communication and external notifications. The system triggers events during the evaluation lifecycle and dispatches them to registered listeners or external webhook endpoints. This mechanism ensures that external systems receive real-time updates without blocking the primary evaluation engine.
Event System Architecture
The core of the notification system relies on an internal event bus. When a significant action occurs in the platform, such as the completion of an evaluation or a change in job status, the system emits an event.
Event Flow
The following diagram illustrates how an internal state change propagates through the system to become an external webhook notification.
The Evaluation Engine triggers events that the Internal Event Bus routes to various consumers. Local listeners handle internal state updates, while the Webhook Worker processes notifications for external integration.
Sources: src/core/events.py:10-45, src/workers/event_handler.py:12-28
Event Types
The platform categorizes events based on the lifecycle of evaluation tasks. Each event carries a specific payload containing relevant metadata and status information.
| Event Name | Description | Trigger Condition |
|---|---|---|
eval.started | Indicates a new evaluation has begun. | When an evaluation job moves to the 'Running' state. |
eval.completed | Signals the successful end of an evaluation. | When all metrics calculations finish. |
eval.failed | Reports an error during execution. | When an unhandled exception occurs or timeout is reached. |
data.ingested | Confirms new dataset availability. | When a file upload or sync process completes. |
Sources: src/core/events.py:50-85, src/db/models.py:102-115
Webhook Implementation
Webhooks allow users to subscribe to specific events via HTTP POST requests. The system supports HMAC signatures to ensure payload integrity and authenticity.
Webhook Configuration
Users configure webhooks by providing a target URL and selecting the events they wish to monitor.
The WebhookSubscription model stores the destination and security credentials, while the EventPayload represents the structured data sent to the subscriber.
Sources: src/db/models.py:140-165, src/api/webhooks.py:20-35
Security and Validation
To secure webhook deliveries, the platform calculates an HMAC SHA-256 signature of the request body using the subscription's secret_key. You must validate this signature in the X-Hub-Signature header to confirm the request originated from the eval-platform.
- Retrieve the raw request body.
- Compute the HMAC using your secret key.
- Compare your result with the header value.
Sources: src/api/webhooks.py:45-60
Event Listeners
Internal listeners perform background tasks that do not require immediate user feedback. This includes logging, metric aggregation, and database cleanup.
Registration Process
Listeners register with the EventDispatcher using a decorator or direct function call.
# Example of a listener registration
@event_bus.on("eval.completed")
def update_user_stats(payload):
# Logic to increment evaluation counts
pass
Sources: src/core/events.py:90-105
Delivery Retries
If a webhook destination returns a non-2xx status code, the worker implements an exponential backoff strategy. The system attempts to redeliver the payload up to five times before marking the delivery as failed in the audit log.
Sources: src/workers/event_handler.py:40-55, src/db/models.py:170-185
The Webhooks & Event Listeners module provides the necessary infrastructure for extensible integrations. By separating event production from consumption, the platform maintains high performance and reliability across different execution environments.