Artifact Indexing & Compound Learning
Relevant source files
The following files were used as context for generating this wiki page:
Artifact Indexing and Compound Learning represent the "Context Graph" system within Continuous Claude. This system is designed to preserve architectural decisions, task outcomes, and session goals in a local, searchable database. By indexing ephemeral markdown files into a structured SQLite format, the system enables Claude to perform RAG-enhanced reasoning based on previous successes and failures.
The system's primary goal is to solve context loss during session clears. Instead of relying on lossy summarization, it builds a persistent memory of the codebase's evolution. This allows for "Compound Learning," where the results of past queries and task post-mortems inform future implementation plans.
Sources: scripts/artifact_schema.sql:1-12, scripts/artifact_index.py:1-15, init-project.sh:150-165
System Architecture
The architecture consists of a three-tier pipeline: extraction of markdown artifacts, indexing into a relational database with Full-Text Search (FTS5), and a query interface for retrieval.
Data Flow Overview
The following diagram illustrates how markdown artifacts move from the thoughts/ directory into the Context Graph for later retrieval.
The system monitors specific directories (thoughts/shared/handoffs, thoughts/shared/plans) and root-level continuity files to populate the database.
Sources: scripts/artifact_index.py:120-130, scripts/artifact_index.py:182-190, init-project.sh:160-170
Database Schema & Full-Text Search
The system utilizes SQLite with the FTS5 extension to provide high-performance text searching with BM25 ranking. The schema is optimized for four primary artifact types: Handoffs, Plans, Continuity snapshots, and Queries.
Schema Definitions
| Table | Purpose | Key Fields |
|---|---|---|
handoffs | Records completed tasks and post-mortems | task_summary, what_worked, what_failed, outcome |
plans | Design and architectural documents | overview, approach, phases, constraints |
continuity | State snapshots of active sessions | goal, state_now, key_learnings |
queries | Stores past Q&A for compound learning | question, answer, was_helpful |
Sources: scripts/artifact_schema.sql:15-80
FTS5 Configuration and Stemming
The system uses the porter tokenizer for stemming and prefix='2 3' for indexing code identifiers. BM25 weights are applied to prioritize specific fields, such as task_summary over files_modified.
-- Example of BM25 column weights configuration
INSERT OR REPLACE INTO handoffs_fts(handoffs_fts, rank)
VALUES('rank', 'bm25(10.0, 5.0, 3.0, 3.0, 1.0)');
Sources: scripts/artifact_schema.sql:85-100, scripts/artifact_index.py:345-355
Indexing Logic
The indexing process involves parsing structured Markdown. The artifact_index.py script uses regular expressions and header detection to split files into logical components.
Handoff Parsing Sequence
Handoffs are the most complex artifacts, requiring the extraction of frontmatter (dates, status, Braintrust IDs) and specific Markdown sections.
Key Indexing Functions
parse_handoff(file_path): Extracts metadata likeoutcome(mapping 'SUCCESS' to 'SUCCEEDED') and file paths usingextract_files.parse_plan(file_path): Capturesphasesandconstraintsfrom architectural plans.index_single_file(conn, file_path): Used by hooks for immediate indexing upon file creation/update.
Sources: scripts/artifact_index.py:80-105, scripts/artifact_index.py:270-300
Compound Learning & Querying
Compound Learning is achieved by storing successful answers to user questions and linking them back to the artifacts that provided the context.
The Query-Learning Loop
This loop ensures that once a complex architectural question is answered, the reasoning is cached and searchable for future sessions.
Sources: scripts/artifact_query.py, tests/test_artifact_query.py:170-190, scripts/artifact_schema.sql:70-82
Search and Ranking
The query interface uses a unified search across all virtual tables. Results are ranked using BM25, ensuring that documents with keyword matches in titles or summaries appear first.
# From search_handoffs logic in artifact_query
# Priority is given to task_summary and what_worked
results = conn.execute("""
SELECT h.*, rank
FROM handoffs_fts f
JOIN handoffs h ON f.rowid = h.rowid
WHERE handoffs_fts MATCH ?
ORDER BY rank
""", (query_text,)).fetchall()
Sources: tests/test_artifact_query.py:230-260, scripts/artifact_index.py:350-360
Project Initialization
The system is initialized via the init-project.sh script, which sets up the required directory structure and creates the SQLite database using the provided schema.
# Core initialization steps from init-project.sh
mkdir -p "$PROJECT_DIR/.claude/cache/artifact-index"
DB_PATH="$PROJECT_DIR/.claude/cache/artifact-index/context.db"
sqlite3 "$DB_PATH" < "$SCRIPT_DIR/artifact_schema.sql"
Sources: init-project.sh:125-160
The Artifact Indexing system transforms ephemeral session data into a persistent knowledge base, enabling Claude to maintain a "Compound Learning" trajectory over the lifecycle of a software project. This ensures that learnings from failures are not repeated and successful patterns are easily recalled.