Memory & Budget Management

Relevant source files

The following files were used as context for generating this wiki page:

• src/budget.rs
• src/curate.rs
• src/raw.rs
• src/providers_db.rs
• src/consumer/gaps.rs
• src/consumer/manifest.rs

Memory & Budget Management in oboete controls how long-term memory for coding agents is processed, stored, and restricted within financial and technical constraints. The system manages a "window" of session records, curates them into meaningful claims, and ensures that AI provider calls remain within user-defined daily and monthly budget ceilings.

This system operates primarily within the curation phase of the worker process. It transforms raw event logs into structured knowledge while using token estimation and cost tracking to prevent excessive spending on AI services.

Curation Windows and Limits

The system processes records in discrete units called windows. A window represents a contiguous segment of session activity that a curator (an AI model) reads and summarizes.

• Window Size: The system uses a ceiling of 5,000 estimated tokens for an interim window size. This size ensures the window, prompt, and expected answer fit within an 8,000-token limit common in free-tier AI providers.
• Record Bounds: One window covers at most 2,000 records to prevent unbounded memory growth from empty records or tombstones.
• Content Elision: Tool outputs larger than a window are elided. The system records the full size of the output but only passes a marker to the curator.
• Shrinking: When the shrink setting is enabled, tool inputs are cut at 300 characters, and long outputs show only the head and tail (300 characters each).

Sources: src/curate.rs:24-40, src/curate.rs:163-176

Window State Transitions

The following diagram shows how a window of records is processed during curation.

The diagram represents the logic for cutting a device's next window of records based on token and record counts.
Sources: src/curate.rs:125-180

Token and Cost Estimation

The system estimates the cost of curation before making provider calls. It uses a character-based estimation for tokens and tracks historical usage to enforce caps.

Token Estimation

The system estimates tokens based on character counts for different languages:

• English: Approximately 4 characters per token.
• Japanese/CJK: Approximately 1.5 characters per token (rounding up).
• Mixed: Uses the specific weights for each character set.

Sources: src/budget.rs:11-30

Budget Enforcement

The system tracks spending in a dedicated SQLite database (providers.db). It enforces limits at multiple levels:

Limit Type	Scope	Description
Daily Budget	Per Provider	Maximum USD or call count allowed per day for a specific AI model.
Monthly Cap	Global	Total USD allowed for all paid providers in the current month.
Cool-down	Per Provider	Mandatory wait time after a provider fails or hits a rate limit.

Sources: src/budget.rs:43-60, src/providers_db.rs:12-25

Provider Management and Failbacks

Curation attempts are governed by a chain of providers. If one provider is unavailable or over budget, the system attempts the next or waits for a reset.

Curation Attempts

• The system allows 3 attempts to curate a window before skipping it.
• If all providers in a chain are skipped (due to budget, rate limits, or errors), the window enters a Pending state.
• Hold Reasons: A window can be held for "time" (retry interval), "budget" (waiting for a daily/monthly reset), or "owner" (waiting for manual intervention).

Sources: src/curate.rs:650-680, src/providers_db.rs:80-100

Curation Flow Logic

The sequence shows the interaction between the worker, the local budget store, and the external AI provider.
Sources: src/curate.rs:620-660, src/providers_db.rs:120-140

Memory Manifests and Compression

The system manages the "As of" state of memory through manifests, ensuring coding agents receive current context while staying within injection limits.

• Injection Cap: Manifests are capped at 6,000 units (tokens/characters depending on agent) to fit within tool context windows.
• Active Sessions: The system identifies active sessions as those with records within the last 30 minutes.
• Gaps and Missing Records: The system monitors "gaps" where the transcript contains turns that the raw record log missed, providing a health metric for the memory system.

Sources: src/consumer/manifest.rs:25-35, src/consumer/gaps.rs:10-20

Data Structure: Pending Curation

The Pending structure tracks windows that could not be curated immediately.

pub struct Pending {
    pub device: String,
    pub from_seq: i64,
    pub from_offset: Option<i64>,
    pub to_seq: i64,
    pub to_offset: Option<i64>,
    pub reason: String,
    pub hold: String, // "time", "budget", or "owner"
    pub attempts: i64,
    pub next_attempt_at: i64,
    pub since: i64,
    pub prompt: String, // SHA256 of the sent prompt
}

Sources: src/providers_db.rs:80-94

Summary

Memory & Budget Management ensures that oboete's curation process is reliable and cost-effective. By using deterministic window cutting, character-based token estimation, and a persistence-backed budget tracking system, the project protects users from unexpected API costs while maintaining a consistent memory state across coding sessions. Curation follows a strict retry logic with specific hold states, ensuring that transient failures or budget resets are handled without losing memory records.