Feature: Search and Querying

Relevant source files

The following files were used as context for generating this wiki page:

• apps/api/examples/README.md
• apps/api/examples/express-app.js
• apps/api/examples/nodejs-usage.ts
• apps/dashboard/src/pages/Dashboard.tsx
• apps/runtime/crates/ts-addon/src/lib.rs
• README.md

The Kodus Runtime provides advanced search and querying capabilities to retrieve persisted events and monitor the system's state. This functionality is exposed through both a Node.js/TypeScript native addon and a REST API, allowing developers to filter events based on specific criteria and gather system statistics. The primary search mechanism is based on the event's source, enabling efficient retrieval of all events originating from a particular service or component.

This system is fundamental for debugging, monitoring, and building analytics on top of the event data stored within the DurableStore. It also includes endpoints for retrieving high-level statistics, such as the total number of events, which is utilized by the monitoring dashboard.

Sources: README.md:10, apps/api/examples/README.md:20-22

Architecture Overview

The search and query functionality spans from the client-facing API down to the core Rust storage engine. A request to query data follows a path through the Node.js API layer, which calls the native Rust addon (ts-addon). The addon then interacts with the DurableStore to perform the actual data retrieval.

The following diagram illustrates the data flow for a typical search request.

This diagram shows the flow of a search query from the end-user to the underlying data store and back.

Sources: apps/api/examples/express-app.js:33-53, apps/runtime/crates/ts-addon/src/lib.rs:98-115

Core Querying Components

The search functionality is primarily managed by the KodusRuntime class, which is a native addon bridging Node.js and the Rust core.

Sources: apps/runtime/crates/ts-addon/src/lib.rs:8-14

KodusRuntime Class

The KodusRuntime class exposed to Node.js provides the main interface for interacting with the storage engine. It includes methods for both searching events and retrieving statistics.

This diagram shows the search-related methods available on the KodusRuntime class and the data structures they return.

Sources: apps/runtime/crates/ts-addon/src/lib.rs:8-14, 98, 117

Key Functions

findEventsBySource(source: String)

This asynchronous function is the primary method for retrieving events. It takes a source string as an argument and returns an array of EventData objects that match that source.

• Implementation: This function is defined in the Rust native addon and calls the DurableStore::find_by_source method internally.
• Return Value: It returns a Promise that resolves to an array of EventData objects.

Sources: apps/runtime/crates/ts-addon/src/lib.rs:98-115, README.md:204-205

getStats()

This asynchronous function retrieves statistics about the DurableStore.

• Implementation: It calls the DurableStore::stats method in the Rust core.
• Return Value: It returns a Promise that resolves to a Stats object containing totalEvents and treeStats.

Sources: apps/runtime/crates/ts-addon/src/lib.rs:117-127, apps/api/examples/README.md:65-66

API Endpoints

The search and stats functionalities are exposed via a REST API, demonstrated with an Express.js example. All critical routes, including search, require JWT authentication.

Sources: README.md:46, apps/api/examples/express-app.js

Method	Endpoint	Description	Authentication
GET	/events/:source	Fetches all events matching the specified source.	Required
GET	/stats	Retrieves statistics from the event store, such as the total event count.	Required
GET	/api/events/stats	The endpoint used by the dashboard to fetch statistics.	Required

Sources: apps/api/examples/express-app.js:33, 55, apps/dashboard/src/pages/Dashboard.tsx:8, README.md:47

GET /events/:source

This endpoint allows clients to query for events by their source.

Request Flow:

This sequence diagram shows the interaction between components when a client requests events by source.

Sources: apps/api/examples/express-app.js:33-53

Example Usage:

curl http://localhost:3201/events/payment-service \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Sources: README.md:40-42, apps/api/examples/README.md:83-84

Data Structures

EventData

This is the structure returned for each event from a search query.

Field	Type	Description
id	string	The unique identifier of the event (UUID).
payload	Buffer	The binary payload of the event.
source	string	The service or component that originated the event.
priority	number	Priority level (0=Low, 1=Normal, 2=High, 3=Critical).
userId	string (optional)	The ID of the user associated with the event.
correlationId	string (optional)	A UUID for tracking the event across services.
tags	[string, string][]	A list of key-value pairs for metadata.
timestamp	string	The ISO 8601 timestamp of when the event was created.
retryCount	number	The number of times processing has been retried.

Sources: apps/runtime/crates/ts-addon/src/lib.rs:177-187, apps/api/examples/README.md:98-109

Stats

This structure contains high-level statistics about the event store.

Field	Type	Description
totalEvents	number (i64)	The total number of events in the store.
treeStats	string (optional)	Internal statistics from the Sled database tree.

Sources: apps/runtime/crates/ts-addon/src/lib.rs:189-193, apps/api/examples/README.md:111-114

Client-Side Usage

Node.js / TypeScript Example

The KodusRuntime addon can be used directly in Node.js applications to query events.

// apps/api/examples/nodejs-usage.ts

// ... initialization
const runtime = new KodusRuntime('./data');

// 5. Buscar e analisar eventos
console.log('\n🔍 Analisando eventos...');

const paymentServiceEvents = await runtime.findEventsBySource('payment-service');
const userServiceEvents = await runtime.findEventsBySource('user-service');

console.log(`📊 Estatísticas:`);
console.log(`  - Eventos de pagamento: ${paymentServiceEvents.length}`);
console.log(`  - Eventos de usuário: ${userServiceEvents.length}`);

This snippet demonstrates how to fetch events by source and use the results for analysis.
Sources: apps/api/examples/nodejs-usage.ts:80-88

Dashboard Integration

The Kodus Dashboard uses the stats-related query functionality to display metrics. It periodically fetches the total event count from the /api/events/stats endpoint to provide a real-time view of the system's activity.

// apps/dashboard/src/pages/Dashboard.tsx

export default function Dashboard() {
    const { data: stats, isLoading, error } = useQuery<Stats>({
        queryKey: ['stats'],
        queryFn: async () => {
            const response = await fetch('/api/events/stats')
            // ... error handling
            return response.json().then(data => data.stats)
        },
        refetchInterval: 5000, // Refetch every 5 seconds
    })

    // ... rendering logic
    <dd className="text-lg font-medium text-gray-900">
        {stats?.totalEvents || 0}
    </dd>
}

This React component shows how react-query is used to poll the stats endpoint and display the totalEvents count.
Sources: apps/dashboard/src/pages/Dashboard.tsx:6-14, 42-44

Summary

The search and querying feature is a core part of Kodus Runtime, providing essential tools for observability and data retrieval. By exposing functionality through both a native Node.js addon and a secure REST API, it allows flexible integration for various use cases, from command-line debugging to real-time monitoring in a web dashboard. The ability to filter events by source and retrieve store-wide statistics makes it a powerful component for managing and understanding the event-driven system.