Performance & Optimization
Relevant source files
The following files were used as context for generating this wiki page:
Ethos employs a multi-tiered approach to performance and optimization, heavily inspired by neuromorphic principles. The system optimizes for both immediate responsiveness (low-latency ingest) and long-term storage efficiency (memory pruning and consolidation). By utilizing background processing, asynchronous pipelines, and specialized database indexing, Ethos ensures that memory operations do not block the critical path of agent-user interaction.
Key optimization strategies include the use of pgvector HNSW indexes for fast semantic retrieval, an async embedding pipeline to prevent API latency from affecting conversation flow, and a consolidation engine that promotes episodic events to durable semantic facts only when necessary.
Sources: README.md:16-30, README.md:43-50, ethos-server/src/subsystems/consolidate.rs:1-15
Database & Vector Retrieval Optimization
Ethos leverages PostgreSQL with the pgvector extension to perform high-performance semantic searches. The primary optimization for retrieval is the Hierarchical Navigable Small World (HNSW) index, which provides approximate nearest neighbor (ANN) search capabilities.
HNSW Index Configuration
The system uses 768-dimensional embeddings (specifically for Gemini models) and applies a vector_cosine_ops index. This configuration is tuned for high-speed similarity matching while maintaining build quality through specific construction parameters.
| Parameter | Value | Description |
|---|---|---|
| Dimensions | 768 | Fixed for Gemini gemini-embedding-001 |
| Index Type | HNSW | Approximate Nearest Neighbor |
| Operator | vector_cosine_ops | Optimized for cosine distance |
m | 16 | Max connections per node |
ef_construction | 64 | Build quality/speed trade-off |
Sources: migrations/001_initial_schema.sql:160-185, ethos-core/src/config.rs:37-45
Trigram Text Search
In addition to vector similarity, Ethos optimizes keyword and pattern matching using the pg_trgm extension. Trigram GIN indexes are applied to content and statement columns to accelerate non-semantic filtering and fact extraction.
Sources: migrations/001_initial_schema.sql:13-14, migrations/001_initial_schema.sql:81-82, migrations/001_initial_schema.sql:123
Background Processing & System Idle Logic
To maintain system responsiveness, heavy computational tasks are deferred to background loops. The Consolidation Engine is a primary example, which runs on a periodic interval (defaulting to 15 minutes) but only if the system is determined to be "idle."
Idle State Detection
The system avoids resource contention by checking two primary metrics before initiating optimization cycles:
- Recent Activity: Queries the
session_eventstable for any records created within a configurableidle_threshold_seconds. - CPU Load: On Linux systems, it reads
/proc/loadavgto ensure the 1-minute CPU load is below a definedcpu_threshold_percent.
The diagram shows the logic gate used to prevent background optimization tasks from interfering with active agent workloads.
Sources: ethos-server/src/subsystems/consolidate.rs:72-85, ethos-server/src/subsystems/consolidate.rs:109-140
Memory Lifecycle Optimization (Decay & Pruning)
Ethos optimizes its storage footprint and retrieval relevance through a neuromorphic "forgetting" mechanism based on Ebbinghaus decay. This prevents the database from growing indefinitely with low-relevance "noise."
The Decay Formula
Salience is calculated and updated in-place during background sweeps. The formula incorporates frequency of access and emotional tone to prioritize important memories:
S(t) = S_0 * e^(-t/τ_eff) * (1 + α*f) * (1 + β*E)
| Variable | Config Name | Function |
|---|---|---|
τ_eff | base_tau_days | Base decay rate (LTP-adjusted) |
α | frequency_weight | Boost for frequently accessed items |
β | emotional_weight | Boost for high emotional tone |
| Threshold | prune_threshold | Soft-delete limit (default 0.05) |
Sources: ethos-server/src/subsystems/decay.rs:3-15, ethos-server/src/subsystems/decay.rs:98-125, ethos-core/src/config.rs:90-96
Long-Term Potentiation (LTP) Optimization
Each time a memory is retrieved, its retrieval_count increases, which exponentially extends its effective time constant (τ_eff). This ensures that frequently used information becomes computationally permanent while unused information is pruned.
This sequence illustrates how successful retrievals optimize the future salience of a memory item.
Sources: ethos-server/src/subsystems/decay.rs:65-110, README.md:58-65
API Performance & Graceful Degradation
The Ethos HTTP API (implemented via Axum) is designed for high concurrency and fail-safety.
- Async Embedding Pipeline: Ingest operations are acknowledged instantly (
queued: true). Embedding generation via the Gemini API occurs in a background Tokio task to prevent external API latency from blocking the chat pipeline. - Graceful Shutdown: All server components (IPC, HTTP, and Consolidation) listen for broadcast shutdown signals to ensure atomic database transactions are completed before the process exits.
- Fire-and-Forget Ingest: The ingest hooks are designed to be non-blocking. If the Ethos server is down, the client-side hooks fail silently with empty-file fallbacks, ensuring agent availability is never compromised by memory service performance issues.
Sources: ethos-server/src/http.rs:48-63, ethos-server/src/http.rs:165-180, README.md:43-50, ethos-server/src/main.rs:94-115
Configuration Summary
Developers can tune performance via ethos.toml using the following optimization parameters:
| Section | Option | Default | Impact |
|---|---|---|---|
database | max_connections | 10 | DB connection pool size |
embedding | batch_size | 16 | Number of vectors to embed in one API call |
consolidation | interval_minutes | 15 | Frequency of episodic facts promotion |
retrieval | confidence_gate | 0.12 | Minimum score to return results |
decay | prune_threshold | 0.05 | Point at which memory is soft-deleted |
Sources: ethos-core/src/config.rs:10-100, README.md:195-205
Summary: Ethos optimizes performance by isolating heavy IO (embeddings) and heavy compute (graph links/consolidation) into background tasks, while using specialized database indexes and system idle checks to ensure that the core memory engine remains fast and non-intrusive for real-time AI agents.