Data Flow: The Lifecycle of an Event
Relevant source files
The following files were used as context for generating this wiki page:
The event is a fundamental data unit in the Kodus Runtime system. Its lifecycle encompasses creation, persistence, retrieval, and processing. This document outlines the end-to-end data flow of an event, from its ingestion through the Node.js API layer down to the Rust core for durable storage and potential execution within a workflow.
The system is designed with a clear separation between the API layer (Node.js/Elysia) and the core processing logic (Rust). A native Node.js addon serves as a high-performance bridge, allowing JavaScript to interact directly with the Rust runtime, which manages event persistence and workflow execution.
1. Event Ingestion via API
Events enter the system through a REST API endpoint. The primary endpoint for creating events is POST /events. This endpoint is responsible for receiving event data, validating it, and passing it to the underlying Rust runtime for persistence.
Sources: apps/api/src/routes/events.ts:9-52
API Endpoint: POST /events
This endpoint requires authentication and accepts a JSON body containing the event's details.
| Parameter | Type | Description | Required |
|---|---|---|---|
payload | object | The main data of the event. | Yes |
source | string | A string identifying the origin of the event (e.g., 'payment-service'). | Yes |
priority | number | A numeric priority level (0-3). Defaults to 1 (Normal). | No |
userId | string | An identifier for the user associated with the event. | No |
correlationId | string | A UUID to group related events. | No |
tags | Array<[string, string]> | An array of key-value pairs for metadata. | No |
Sources: apps/api/src/routes/events.ts:16-24
The following diagram shows the sequence of an event being created via the API.
2. The Rust Native Addon Bridge
The kodus-runtime Node.js package is a native addon written in Rust. It exposes the KodusRuntime class, which acts as the bridge between the JavaScript environment and the Rust core logic. When the API receives a request, it calls methods on an instance of this class.
The persistEventWithMetadata function is a key part of this bridge. It takes JavaScript data types, converts them into the corresponding Rust Event struct, and invokes the DurableStore to persist it.
Sources: apps/runtime/crates/ts-addon/src/lib.rs:7-22, apps/runtime/crates/ts-addon/src/lib.rs:40-75
This diagram illustrates the data conversion from the Node.js layer to the Rust core.
3. Core Event Structure
At the heart of the system is the Rust Event struct, which provides a strongly-typed representation of an event and its associated metadata. The Node.js addon exposes a simplified version of this, EventData, for communication with the JavaScript world.
Sources: apps/runtime/crates/core/src/queue.rs:13-39, apps/runtime/crates/ts-addon/src/lib.rs:168-179
Event and EventData Relationship
The diagram below shows the core Event struct and its relationship with the EventData object used in the Node.js addon.
Event Priority
Events can be assigned a priority, which can be used by processing queues to handle more important events first.
| Numeric Value | Enum Variant | Description |
|---|---|---|
| 0 | Low | Low priority. |
| 1 | Normal | Default priority. |
| 2 | High | High priority. |
| 3 | Critical | Critical, urgent priority. |
Sources: apps/runtime/crates/ts-addon/src/lib.rs:50-55, apps/runtime/crates/core/src/queue.rs:42-48
4. Durable Persistence
Once an Event struct is created in the Rust core, it is passed to the DurableStore for persistence. The DurableStore is responsible for writing the event to disk, ensuring that it is not lost in case of an application restart or crash. The underlying storage engine is Sled, a high-performance embedded database.
The primary method for this is persist_event, which takes a reference to an Event and returns its generated ID.
Sources: apps/runtime/crates/core/src/store.rs:43-46, README.md
5. Event Retrieval
Events can be retrieved from the store using various criteria. The API exposes a GET /events/:source endpoint, which allows clients to fetch all events originating from a specific source.
This request flows from the API through the ts-addon's findEventsBySource function, which in turn calls the DurableStore's find_by_source method in the Rust core. The results are then converted back into EventData objects for the JavaScript client.
Sources: apps/api/src/routes/events.ts:55-77, apps/runtime/crates/ts-addon/src/lib.rs:78-94, apps/runtime/crates/core/src/store.rs:53-61
6. Event Processing with Workflows
Beyond simple storage and retrieval, an event can be the input for a complex business process defined as a Workflow. The WorkflowEngine is responsible for executing a series of steps (e.g., Transform, Filter, Route) on an event.
An event's lifecycle can be extended by passing it to a workflow via the POST /workflows/:workflowId/execute API endpoint or the execute_workflow function in the native addon. The engine processes the event according to the workflow definition and returns the final, processed event.
The diagram below shows the high-level flow of an event through the WorkflowEngine.
Conclusion
The lifecycle of an event in Kodus Runtime is a robust, multi-layered process. It begins with a simple API call, traverses a high-performance Rust native addon, is represented as a strongly-typed Rust struct, and is durably persisted to disk. From there, it can be retrieved for auditing or analytics, or it can be fed into a powerful workflow engine for complex, stateful processing. This architecture ensures both high performance and data integrity throughout the event's journey.