Workflows & Background Jobs
Relevant source files
The following files were used as context for generating this wiki page:
The Memory Context Browser (MCB) utilizes a structured approach to managing complex, multi-stage operations and asynchronous tasks. These are primarily categorized into Workflows, which represent high-level logical sequences (such as semantic search or project management), and Background Jobs, which handle resource-intensive tasks like repository indexing.
Workflows in MCB are designed to provide AI coding agents with deep codebase understanding through persistent memory and semantic search. These processes often involve cross-session observation storage, context injection, and state management via Project and Entity families of operations.
Sources: README.md:10-18, AGENTS.md:155-165
Workflow Architecture and Lifecycle
Workflows in MCB follow a multi-phase lifecycle to transform raw codebase data into searchable semantic information or structured project states. A primary example is the Semantic Search Workflow, which transitions through parsing, state management, embedding generation, and final search execution.
Semantic Search Workflow Phases
- Parsing and Chunking: Source code is parsed using Tree-sitter into language-aware chunks.
- State Management: Chunks are grouped into snapshots representing the codebase at a specific point in time.
- Embedding Generation: Chunks are processed by embedding providers to generate high-dimensional vectors.
- Search & Ranking: Natural language queries are embedded and compared against stored vectors to return ranked results.
Sources: crates/mcb-domain/tests/integration/semantic_search_workflow.rs:16-65
The diagram shows the sequential stages of the semantic search workflow, from initial parsing to the final delivery of ranked search results.
Sources: crates/mcb-domain/tests/integration/semantic_search_workflow.rs:13-75
Project & Task Orchestration
Workflows are also applied to project management, where ProjectHandler manages the lifecycle of project entities including phases, issues, and dependencies. This system supports finite-state machine (FSM) based task orchestration to enforce policies and track progress across complex developer tasks.
Sources: README.md:105-107, crates/mcb-server/src/handlers/project.rs:25-50
Background Jobs and Indexing
Background jobs handle long-running, non-blocking tasks. The most critical background job is the Repository Indexer, which processes codebases to prepare them for semantic search.
Indexing Job Process
The indexing job is triggered via the index_repo MCP tool. It involves:
- Clearing stale indices to prevent duplicate or conflicting data.
- Scanning files based on configured extensions (e.g.,
.rs,.ts,.py). - Incremental indexing based on file hashes to optimize performance.
Sources: tests/e2e/browse-ui.spec.ts:40-65
This sequence illustrates the asynchronous nature of indexing jobs, where the server acknowledges the request while the actual processing occurs in a background worker.
Sources: tests/e2e/browse-ui.spec.ts:15-60, tests/e2e/admin-functional.spec.ts:40-45
Job Monitoring and Statuses
The system provides visibility into the status of background jobs through an administrative interface and dedicated API endpoints.
| Metric | Description |
|---|---|
| Total | Cumulative number of jobs processed. |
| Running | Current number of active background processes. |
| Queued | Jobs waiting for available processing resources. |
| Status | Visual state: idle, running, or complete. |
Sources: tests/e2e/admin-functional.spec.ts:40-45, tests/e2e/admin-functional.spec.ts:85-95
Integration and Tooling
Workflows and jobs are exposed to the Model Context Protocol (MCP) through specific operation families.
- Index Family:
index_repo,index_status,clear_index. - Project Family: Manages
Project,Phase,Issue,Dependency, andDecisionresources. - Search Family:
search_code,search_memory.
Project Workflow Actions
The ProjectHandler routes various actions that drive the project workflow:
| Resource | Supported Actions |
|---|---|
| Project | Create, Get, List, Update, Delete |
| Phase | Create, Get, List, Update, Delete |
| Issue | Create, Get, List, Update, Delete |
| Dependency | Create, List, Delete |
| Decision | Create, Get, List, Update, Delete |
Sources: README.md:70-85, crates/mcb-server/src/handlers/project.rs:40-48
Summary
Workflows and background jobs form the operational backbone of the Memory Context Browser. By separating high-level logic (Workflows) from intensive data processing (Background Jobs), MCB ensures that AI agents can interact with massive codebases efficiently. The use of snapshots for state management and hashing for incremental indexing allows the system to maintain a performant, persistent memory of code context.
Sources: README.md:10-25, crates/mcb-domain/tests/integration/semantic_search_workflow.rs:13-20