Manifest & File Ingestion

Relevant source files

The following files were used as context for generating this wiki page:

• src/manifest.rs
• src/consumer/manifest.rs
• src/import.rs
• src/digest.rs
• src/curate.rs
• src/main.rs

Manifest and File Ingestion components manage the context handed to coding agents at the start of a session. The system extracts relevant "facts" from raw events—such as git state, file paths, and user directives—and renders them into a deterministic text manifest.

The ingestion process involves scanning event logs to identify touched files, failing commands, and pending todo items. This data is stored in a structured format within a knowledge database (knowledge.db) to ensure agents receive concise, actionable context from previous sessions.
Sources: src/manifest.rs:1-6, src/consumer/manifest.rs:1-10

Manifest Architecture

The manifest is a deterministic text summary of the current state of a repository checkout. It consists of multiple sections that are dropped in a specific priority order if the total length exceeds a character cap.
Sources: src/manifest.rs:104-124, src/consumer/manifest.rs:24-35

Data Structures

The system uses specific structures to represent the components of a manifest.

Structure	Purpose
Line	Represents a single prompt line with its date, session ID, and text content.
Parts	Holds all logical sections of a manifest before they are rendered into text.
Manifest	Consumer struct that handles the lifecycle of manifest data in the knowledge database.

Sources: src/manifest.rs:8-14, src/manifest.rs:69-83, src/consumer/manifest.rs:37-39

Manifest Rendering Logic

The render function constructs the final text by concatenating sections. If the combined text exceeds the CAP (default 6,000 characters), sections are dropped from the end of the list until the text fits.
Sources: src/manifest.rs:104-124, src/consumer/manifest.rs:24-25

The diagram above shows the priority order of sections; sections at the top (like Git State) are preserved while sections at the bottom (like Active Sessions) are dropped first to meet the length limit.
Sources: src/manifest.rs:104-136, src/manifest.rs:222-231

File Ingestion and Fact Extraction

The manifest consumer processes raw events to extract specific "facts" used to build the manifest.

Fact Types and Extraction

The consumer identifies events such as prompt, reply, and tool to populate the manifest_facts table.
Sources: src/consumer/manifest.rs:226-261

• File Paths: Extracted from tool inputs (e.g., file_path, notebook_path) and patch commands. Paths are normalized to use / and are stored relative to the current working directory.
• Failing Commands: Identified by non-zero exit codes or specific failure markers in tool outputs. A command is considered "fixed" if the same command succeeds in a later event.
• Todo Items: Parsed from specific agent tool outputs like Claude Code's todos or Codex's plan.
• Owner Directives: Filtered from user prompts based on specific markers (e.g., "always", "never", "from now on").

Sources: src/consumer/manifest.rs:241-261, src/consumer/manifest.rs:319-354, src/consumer/manifest.rs:357-377

Persistence Schema

The extraction results are stored in the knowledge database using the following table structure:
Sources: src/consumer/manifest.rs:52-78

The manifest_facts table acts as a log of extracted information, while the manifests table stores the final rendered text for a specific checkout.
Sources: src/consumer/manifest.rs:52-78

Directive Processing

Directives are standing instructions from the owner. The system uses a negation-aware logic to determine which directives are currently in force.
Sources: src/manifest.rs:16-65

Processing Flow

1. Identification: A line is identified as an "owner line" if it contains directive markers like "always" or negation markers like "cancel that".
2. Negation Logic: A negation line removes all earlier directive lines that share a common content word (excluding common English stop words).
3. Normalization: Directive text is cleaned of markers and split into words/trigrams to determine what the instruction is about.

Sources: src/manifest.rs:31-65

This logic ensures that saying "cancel the clippy rule" successfully removes a previous instruction like "always run clippy".
Sources: src/manifest.rs:55-65

External Data Ingestion (Claude-mem)

The import module provides functionality to ingest data from the legacy claude-mem tool.

Mapping Logic

The importer maps claude-mem entities to the current project's schema.
Sources: src/import.rs:39-166

Claude-mem Entity	Mapped Entity	Logic
observations	observations	Narrative and facts are joined with bullet points.
session_summaries	summaries	Request, Investigated, Learned, and Completed fields are joined.
user_prompts	prompts	Text is cleaned of private blocks and redacted.
project	repo	Projects are prefixed with claude-mem:.

Sources: src/import.rs:81-166, src/import.rs:247-255

Import Safety

To prevent duplicate data, the system maintains an imports table. It identifies source databases by the applied_at timestamp of the first migration row to ensure that moving or backing up a database does not change its identity.
Sources: src/import.rs:172-192, src/import.rs:198-228

Git State Ingestion

The manifest consumer actively monitors the git state of the checkout to warn agents about risky environments.
Sources: src/consumer/manifest.rs:480-511

Risky States Detected

The system identifies the following conditions using git status --porcelain=v2:

• Progress Indicators: Active rebase, merge, cherry-pick, or revert.
• Head State: Detached HEAD status.
• Change Counts: Number of uncommitted changes, untracked files, and merge conflicts.
• Synchronization: Commits not yet pushed to the remote.

Sources: src/consumer/manifest.rs:480-529

Status Parsing logic

The porcelain function parses git output into human-readable strings.
Sources: src/consumer/manifest.rs:514-539

// Example of git porcelain parsing in manifest consumer
fn porcelain(status: &str) -> Vec<String> {
    let (mut changed, mut conflicts, mut untracked, mut ahead) = (0, 0, 0, 0);
    for line in status.lines() {
        match line.split_once(' ') {
            Some(("1" | "2", _)) => changed += 1,
            Some(("u", _)) => conflicts += 1,
            Some(("?", _)) => untracked += 1,
            // ... parsing continues
        }
    }
    // Returns formatted strings like "2 uncommitted change(s)"
}

Sources: src/consumer/manifest.rs:514-529

Summary

The Manifest & File Ingestion system provides the essential context required for coding agents to function effectively within a specific repository. By deterministic rendering of git states, failing commands, and standing directives, the system ensures that agents are aware of recent history and owner preferences. Ingestion from external sources and the extraction of facts from raw event logs allow the context to evolve naturally as the user interacts with the coding agents.