Retrieval & Reranking Model
Relevant source files
The following files were used as context for generating this wiki page:
The Retrieval & Reranking Model in cxs is designed for the progressive discovery of local Codex session logs. Rather than loading entire transcripts, the system follows a multi-stage pipeline: recalling candidate sessions through Full-Text Search (FTS), applying heuristic reranking to prioritize relevant context, and finally providing snippets that guide the user to specific message ranges or pages.
Sources: README.md:158-160, AGENTS.md:20-22
Retrieval Architecture
The retrieval process is structured as a pipeline that moves from broad recall to specific session prioritization. The system leverages SQLite's FTS5 engine for initial matching and a custom heuristic layer for ranking.
The diagram shows the flow from a raw user query through tokenization, multi-table FTS recall, heuristic scoring, and final formatting.
Sources: AGENTS.md:20-28, skill-packages/cxs/references/advanced-queries.md:3-10
Stage 1: Recall & Search Surface
Candidates are recalled from two primary FTS virtual tables: messages_fts and sessions_fts.
Search Surfaces
- Message Level: Indexed over real user and assistant messages stored in the
messagestable. - Session Level: Indexed over metadata including the title,
summary_text,compact_text, andreasoning_summary_text. - CJK Fallback: For queries containing CJK (Chinese, Japanese, Korean) characters where tokenization results in zero tokens, the system falls back to a bounded
LIKEscan.
Sources: AGENTS.md:33-35, README.md:164-167, skill-packages/cxs/references/advanced-queries.md:18-25
FTS Field Weights
During session-level recall, specific fields are weighted to prioritize matches in more descriptive metadata.
| Field | Weight | Description |
|---|---|---|
title | 8.0 | The initial user query or derived session name. |
compact_text | 4.0 | Compressed context from session handoffs. |
summary_text | 3.0 | Derived summary of first/last user/assistant turns. |
reasoning_summary_text | 1.2 | Internal reasoning steps or summaries. |
Sources: AGENTS.md:37-38, src/parser.ts:108-120
Stage 2: Heuristic Reranking
The rerankHits logic aggregates multiple hits for the same session and applies a series of heuristics to calculate the final relevance score.
Scoring Logic
- Normalization: Raw FTS scores (BM25) are typically negative in SQLite; the system normalizes these so that higher values represent better matches.
- Grouping: Rows are grouped by
sessionUuidto provide a session-centric view. - Boosts:
- User Role: Matches in messages with
matchRole: 'user'receive a score bonus. - Title/CWD Match: Direct matches between query terms and the session title or Current Working Directory (CWD) provide significant boosts.
- User Role: Matches in messages with
- Tie-breaking: When scores are equal, the system prefers recency based on the
endedAttimestamp.
Sources: src/ranking.test.ts:44-55, src/ranking.test.ts:57-67, src/ranking.test.ts:79-87, src/ranking.test.ts:98-106
Data Extraction Flow
The reranking depends on high-quality metadata extracted during the sync phase by the parser.
This diagram illustrates how the parser extracts specific record types to build the searchable metadata used by the reranking model.
Sources: src/parser.ts:45-80, AGENTS.md:30-31
Stage 3: Result Projection & Guidance
The final output of the retrieval model is a FindResult object, which includes a snippet and determines the "Next Step" for the user.
Match Attribution
The system distinguishes between results found in specific messages versus general session metadata.
| Match Source | matchSeq | Guidance |
|---|---|---|
message | Integer | Suggests read-range --seq <matchSeq> to view context. |
session | null | Suggests read-page to explore the session from the start. |
Sources: README.md:169-171, skill-packages/cxs/references/json-schema.md:14-29
Snippet Generation
Snippets are generated by makeRawSnippet or makeLikeSnippet. The logic centers the snippet around the match anchor, providing roughly 40 characters before and 80 characters after the hit. Matches are wrapped in <mark> tags for UI highlighting.
Sources: src/query/snippet.ts:1-15, src/query/snippet.ts:17-30
// Example snippet logic from src/query/snippet.ts:2-7
const index = lower.indexOf(target);
if (index < 0) return content.slice(0, 160);
const start = Math.max(0, index - 40);
const end = Math.min(content.length, index + target.length + 80);
const snippet = content.slice(start, end);
Sources: src/query/snippet.ts:2-7
Summary
The cxs Retrieval & Reranking Model prioritizes precision by combining standard FTS recall with domain-specific heuristics (CWD and Role boosts). By categorizing matches into message or session sources, the model enables a progressive workflow where users move from a global search to a local message range exploration.