Server Subsystems
Relevant source files
The following files were used as context for generating this wiki page:
Ethos server is architected as a set of modular subsystems that coordinate to provide long-term neuromorphic memory for AI agents. The server handles high-performance ingestion, semantic search, and background "biological" processes such as memory consolidation and decay. These subsystems interface with a PostgreSQL database equipped with pgvector for high-dimensional embedding storage and retrieval.
The core responsibility of the server is to manage the lifecycle of a memory, transitioning it from a raw session event into an episodic trace, and eventually consolidating it into a durable semantic fact. This mimics the human hippocampus's role in converting short-term experiences into long-term knowledge.
Sources: README.md:1-15, ethos-server/src/main.rs:1-10
Subsystem Overview
The Ethos server orchestrates several specialized subsystems, each running as independent asynchronous tasks or responding to specific IPC/HTTP requests.
| Subsystem | Responsibility | Trigger Mechanism |
|---|---|---|
| Ingest | Records raw conversation turns and session metadata into session_events. | IPC/HTTP Request |
| Embedder | Generates 768-dim Gemini embeddings for incoming content asynchronously. | Background Loop |
| Retrieve | Performs semantic search using pgvector and spreading activation. | IPC/HTTP Request |
| Consolidation | Promotes episodic memories to structured semantic facts (decisions, preferences). | 15-min Idle Loop / Manual |
| Decay | Implements Ebbinghaus decay to prune or strengthen memories over time. | Post-Consolidation |
Sources: README.md:120-145, ethos-server/src/main.rs:77-118
System Architecture Flow
The following diagram illustrates how data flows through the various subsystems from initial ingestion to long-term storage.
The diagram shows the modular separation between request handling, background processing, and the database layer.
Sources: README.md:120-155
Consolidation Subsystem
The Consolidation Subsystem implements the "Default Mode Network" (DMN) logic. It is responsible for scanning episodic_traces and promoting high-value information into semantic_facts. This process typically runs every 15 minutes when the system detects it is idle.
Promotion Criteria
Memories are promoted based on specific salience scores and content markers:
- Importance: Must be $\ge$ threshold (default 0.8).
- Retrieval Frequency: Accessed $\ge$ 5 times.
- Decision Keywords: Contains phrases like "we decided", "the plan is", or "let's go with".
- Preference Keywords: Matches patterns for "prefers", "loves", "hates", "favorite".
- Explicit Markers: Content starting with "Remember this:" or "Important:".
Sources: ethos-server/src/subsystems/consolidate.rs:1-20, ethos-core/src/config.rs:55-65
Conflict Resolution
When consolidating a new fact that contradicts existing knowledge, the subsystem applies a tiered resolution strategy:
- Refinement: If the subject/predicate match and objects are compatible, the existing fact is updated with new context.
- Supersession: Explicit decisions always supersede old facts. High-confidence deltas ($> 0.15$) also trigger automatic supersession.
- Flagging: If a contradiction is ambiguous, both facts are flagged for human review in a Markdown-based review inbox.
Sources: ethos-server/src/subsystems/consolidate.rs:10-15, ethos-server/src/subsystems/consolidate.rs:320-370
HTTP Subsystem
The HTTP subsystem provides a REST API via the Axum framework, running on port 8766 by default. It acts as a bridge to the internal IPC router, allowing standard web clients and the ethos-cli to interact with the memory engine.
API Endpoints
| Endpoint | Method | Description |
|---|---|---|
/health | GET | Returns DB connection status and pgvector version. |
/search | POST | Performs semantic memory search with optional spreading activation. |
/ingest | POST | Ingests content into the asynchronous embedding pipeline. |
/consolidate | POST | Manually triggers a consolidation cycle. |
Sources: ethos-server/src/http.rs:15-35, ethos-server/src/http.rs:100-150
Request Processing Sequence
The HTTP subsystem delegates business logic to "inner" functions to ensure testability and separation of concerns.
Embedding Subsystem
The Embedding subsystem manages the asynchronous pipeline for generating vectors. Ethos uses 768-dimensional Gemini embeddings by default.
Configuration Options
| Parameter | Type | Default | Description |
|---|---|---|---|
backend | String | "gemini" | Embedding provider (gemini or onnx). |
batch_size | u32 | 16 | Number of items to embed in a single API call. |
rate_limit_rpm | u32 | 100 | Maximum requests per minute to the embedding API. |
queue_capacity | u32 | 1000 | Buffer size for pending embedding tasks. |
Sources: ethos-core/src/config.rs:35-48, README.md:130-140
Decay Engine Subsystem
Directly following a consolidation cycle, the Decay Subsystem executes a sweep to update the salience of all stored memories based on the Ebbinghaus Forgetting Curve.
Mathematical Model:
The effective time constant $\tau_{eff}$ is adjusted by Long-Term Potentiation (LTP). Memories that are retrieved frequently have their half-life extended significantly.
- Never Accessed: ~10 days to prune.
- 5+ Retrievals: ~75 days to prune.
- 10+ Retrievals: ~1.5 years to prune.
Sources: README.md:85-115, ethos-server/src/subsystems/consolidate.rs:90-100
Summary
The Ethos server subsystems form a cohesive pipeline that transforms transient conversation data into persistent, structured knowledge. By separating ingestion and retrieval from heavy background tasks like consolidation and decay, the system maintains high responsiveness for real-time AI agent interactions while ensuring long-term data integrity and relevance through neuromorphic principles.