FFF-MCP Tooling Integration
Relevant source files
The following files were used as context for generating this wiki page:
The FFF-MCP (Fast File Finder - Model Context Protocol) tooling integration represents the primary interface for repository exploration and code analysis within the swe-term environment. It establishes a mandatory policy for AI agents to interact with the codebase through structured MCP tools rather than native shell utilities for tasks involving filename discovery, content search, and scoped filtering in git-indexed directories.
This integration is a core component of the "Polyglot Principal Engineer" persona, which prioritizes correctness and observability. By abstracting repository searches through FFF-MCP, the system ensures that AI agents operate within safe boundaries, leveraging specialized tools like ast-grep and demongrep for structural code modifications instead of brittle regex-based shell commands.
Sources: AGENTS.md:149-165, AGENTS.md:167-179, src/prompts.yaml:1-15
Tooling Policy and Hierarchy
The project enforces a strict hierarchy for tool selection to maintain code safety and maintainability. AI agents are instructed to default to FFF-MCP tools for all repository exploration. Shell-native tools like grep, ripgrep (rg), and find are explicitly discouraged for general exploration and are only permitted as fallbacks when FFF-MCP cannot perform a specific task.
Preferred Tool Categories
| Category | Primary Tools | Purpose |
|---|---|---|
| Text Streams | fff tools, grep/rg, sed, awk, jq/yq | Logs, CSV, configs, and simple plaintext search. |
| Source Code | demongrep, osgrep, ast-grep, semgrep, grit | Structural code research, refactors, and audits. |
Sources: AGENTS.md:149-165, AGENTS.md:167-179
Search Execution Flow
The following diagram illustrates the decision logic for tool selection when an agent needs to explore the repository.
The diagram shows the mandatory check for git-indexed directories, favoring FFF-MCP tools as the primary interface.
Sources: AGENTS.md:167-179
Structural Search with ast-grep
A significant portion of the FFF-MCP integration is dedicated to ast-grep, a tool for structural code search and transformation. Unlike regex-based tools, ast-grep understands the Abstract Syntax Tree (AST) of the source code, allowing for precise matching of code patterns, node types (kind), and relational conditions (e.g., code inside a specific function).
Core MCP Tools for ast-grep
- dump_syntax_tree: Dumps the AST of a code snippet, essential for debugging patterns.
- test_match_code_rule: Validates a search rule against a provided code snippet before executing it on the full codebase.
Sources: src/prompts.yaml:1-55, src/prompts.yaml:70-95
Relational Search Logic
When developing search rules, the integration emphasizes relational rules to capture context. A critical requirement is the use of stopBy: end to ensure searches traverse the entire AST direction (root for inside, leaves for has).
This flow ensures that relational rules do not terminate prematurely at immediate neighbors.
Sources: src/prompts.yaml:22-35, src/prompts.yaml:129-145
LLM-Enhanced Extraction Pipeline
The FFF-MCP integration works alongside the swe_distiller module, which provides a Rust-based extraction pipeline for converting web content into markdown. The pipeline supports an optional --llm mode that utilizes providers (e.g., Jina, Ollama with reader-lm or gemma) to clean and structure content.
Distillation Flow
The diagram shows the multi-stage process where LLM output is compared against heuristic baselines using safety gates.
Sources: extensions/swe_distiller/ARCHITECTURE.md:15-25, extensions/swe_distiller/src/lib.rs:27-65, extensions/swe_distiller/src/extraction/llm.rs:26-60
Safety and Quality Gates
LLM-extracted content is subjected to safety checks defined in src/extraction/checks.rs. A candidate must pass word count thresholds (typically within 75% to 135% of the original heuristic extraction) and heading overlap checks to be accepted.
| Check Type | Threshold / Logic | Purpose |
|---|---|---|
| Word Count | 0.75x to 1.35x of baseline | Prevent hallucinated truncation or expansion. |
| Link Count | > 0.55x of baseline (if links >= 6) | Ensure information density is preserved. |
| Noise Score | candidate <= baseline + 2 | Penalize navigation chrome, social links, and digits. |
Sources: extensions/swe_distiller/src/extraction/checks.rs:44-70, extensions/swe_distiller/src/extraction/llm.rs:52-58
Implementation Notes
The system leverages AST-based markdown conversion (src/markdown_ast.rs) to maintain structural integrity during the transformation from HTML. This includes specific handling for headers, code blocks (with language detection via CSS classes), tables, and nested lists.
- Language Detection: The pipeline searches for
language-orhighlight-source-classes within<pre>or<code>tags to provide syntax highlighting in the final markdown output. - Table Handling: The AST walker converts HTML tables into standard markdown pipes and dashes format, preserving headers defined in
<th>tags.
Sources: extensions/swe_distiller/src/markdown_ast.rs:40-80, extensions/swe_distiller/src/markdown_ast.rs:118-145, extensions/swe_distiller/src/markdown_ast.rs:242-265
The FFF-MCP integration ensures that repository interactions are structured, searchable, and safe, providing a robust foundation for AI agents to perform complex software engineering tasks within the swe-term harness.