AST-Grep & Advanced Search
Relevant source files
The following files were used as context for generating this wiki page:
AST-Grep and advanced search mechanisms form the foundation of the project's codebase exploration and refactoring capabilities. These tools allow agents to perform structural code analysis beyond simple text matching by understanding the Abstract Syntax Tree (AST) of the source code. This enables precise pattern matching, previewing transformations, and executing deterministic refactors across repository-scale architectures.
The system integrates these tools into an "Exploration Hierarchy," where direct tools like grep, rg, and ast_grep_search are utilized by autonomous agents such as Hephaestus and Librarian to gather comprehensive context before acting. Sources: src/agents/hephaestus.ts:162-175, src/features/builtin-commands/templates/refactor.ts:88-115
Structural Analysis with AST-Grep
AST-Grep is primarily used for finding and replacing structural patterns within the code. Unlike standard grep, it recognizes syntactic structures like functions, classes, and variable declarations, regardless of formatting or whitespace.
Key Operations
The system utilizes AST-Grep through specific interfaces for search and preview-based replacement:
- Pattern Search: Finding specific code structures (e.g.,
function $NAME($$$) { $$$ }). - Dry Run Replacement: Previewing refactoring changes before applying them to ensure safety.
- Targeted Refactoring: Executing precise changes using
ast_grep_replacebased on validated patterns.
Sources: src/features/builtin-commands/templates/refactor.ts:101-115, src/agents/hephaestus.ts:162-168
Advanced Search Workflow
The following diagram illustrates how the system transitions from general exploration to precise AST-based modification during a refactoring task.
Exploration Hierarchy & Tool Integration
The system employs a multi-layered approach to searching, prioritizing efficiency and precision based on the task requirements. This hierarchy ensures that agents exhaust internal and external resources before requesting user input.
| Tool Category | Specific Tools / Agents | Primary Use Case |
|---|---|---|
| Direct Tools | grep, rg, git log, ast_grep_search | Known locations, exact pattern matches. |
| Explore Agents | explore subagent | Contextual grep for multi-angle internal search. |
| Librarian Agents | librarian subagent | External reference search (docs, OSS examples). |
| LSP Tools | LspFindReferences, LspDocumentSymbols | Precise dependency mapping and symbol lookup. |
Sources: src/agents/hephaestus.ts:162-175, sisyphus-prompt.md:126-155, src/features/builtin-commands/templates/refactor.ts:88-100
Librarian External Search Protocol
The librarian agent specializes in searching remote codebases and documentation when local search fails or unfamiliar libraries are encountered. It follows a specific "Request Classification" to choose the right search strategy.
Implementation in Refactoring Cycles
AST-Grep is a critical component of the refactor command template. The system mandates a "Preview-Before-Act" philosophy to prevent destructive changes.
Refactoring Execution Steps
- Direct Exploration: Use
ast_grep_searchto identify structural targets across the defined scope. Sources: src/features/builtin-commands/templates/refactor.ts:101-105 - Codemap Construction: Mapping the dependencies of the target symbols to identify "Impact Zones". Sources: src/features/builtin-commands/templates/refactor.ts:125-145
- Transformation Preview: Executing
ast_grep_replacewithdryRun=trueto visualize changes. Sources: src/features/builtin-commands/templates/refactor.ts:107-112 - Deterministic Execution: Applying changes only after
lsp_diagnosticsconfirms a clean baseline. Sources: src/features/builtin-commands/templates/refactor.ts:192-210
Structural Pattern Examples
The system defines common patterns for search and replacement to maintain codebase consistency:
// Example from refactor template logic
ast_grep_search(
pattern="function $NAME($$$) { $$$ }",
lang="typescript",
paths=["src/"]
)
// Example replacement logic
ast_grep_replace(
pattern="[old_pattern]",
rewrite="[new_pattern]",
lang="[language]",
dryRun=true
)
Sources: src/features/builtin-commands/templates/refactor.ts:101-112
Project Scale and Dynamic Search Spawning
For large-scale projects, the init-deep command dynamically scales search resources based on codebase complexity thresholds. This ensures thorough coverage during initial discovery phases.
| Factor | Threshold | Action |
|---|---|---|
| Total Files | >100 | +1 Explore agent per 100 files |
| Total Lines | >10k | +1 Explore agent per 10k lines |
| Directory Depth | ≥4 | +2 agents for deep exploration |
| Monorepo | Detected | +1 agent per package/workspace |
Sources: src/features/builtin-commands/templates/init-deep.ts:50-70
Summary
The AST-Grep and Advanced Search system provides a robust framework for autonomous code understanding and modification. By combining structural analysis (AST-Grep), repository-wide mapping (LSP/Grep), and external research (Librarian), the project enables complex refactoring tasks to be executed with high precision and verifiable safety. This multi-layered approach ensures that even deep architectural changes are grounded in a comprehensive understanding of the existing codebase patterns.