Command: code-improvement-scanner
Relevant source files
The following files were used as context for generating this wiki page:
The code-improvement-scanner is a specialized command utilized within the swe-term environment to automate the identification and prioritization of code quality issues, structural noise, and potential refactoring targets. It operates by analyzing source code structures, often leveraging Abstract Syntax Tree (AST) patterns and Large Language Model (LLM) providers to suggest improvements while maintaining strict safety and quality gates.
This command is integrated into the broader agentic workflow, adhering to principal engineer standards such as "correctness & user intent" and "operational simplicity." It utilizes tools like ast-grep for structural analysis and the swe_distiller pipeline for cleaning and standardizing textual representations of code and documentation.
Sources: AGENTS.md:33-50, extensions/swe_distiller/ARCHITECTURE.md:1-10
Core Analysis Engine
The scanner utilizes a multi-layered approach to evaluate code quality, ranging from deterministic structural checks to heuristic noise scoring and LLM-driven deep analysis.
Structural Noise Scoring
A primary component of the scanner's logic is the evaluation of structural noise. This is calculated by penalizing specific patterns that indicate low-quality content or "chrome" (navigation elements, standalone digits, or very short non-heading lines).
The scoring logic specifically targets UI elements like "Sign up", "Search", and "Follow" to ensure the scanner focuses on core logic rather than peripheral noise.
Sources: extensions/swe_distiller/src/extraction/checks.rs:1-35
AST-Based Searching
For deep structural analysis, the scanner employs ast-grep. This allows the tool to search for specific code patterns using metavariables (e.g., $VAR, $$$ARGS) and relational rules like inside or has.
| Rule Property | Purpose | Example |
|---|---|---|
pattern | Matches node by code structure | pattern: console.log($ARG) |
kind | Matches node by AST type | kind: call_expression |
inside | Target must be within a specific parent | inside: { pattern: class $C { $$$ } } |
has | Target must contain a specific descendant | has: { pattern: await $EXPR } |
Sources: src/prompts.yaml:69-82
LLM Integration & Safety Pipeline
When enabled, the scanner routes code blocks through an LLM pipeline for high-level improvement suggestions. To prevent regressions, every LLM output is subjected to a "Safety & Quality Gate."
Safety Gates
The scanner compares the original code/content against the LLM-generated candidate using the passes_safety_checks logic.
- Word Count Stability: Candidates must be within 75% to 135% of the original word count to prevent excessive deletion or hallucination.
- Link Retention: If the original content has 6 or more links, the candidate must retain at least 55% of them.
- Heading Overlap: The scanner calculates token overlap between the first headings; an overlap of less than 50% triggers a safety failure.
Sources: extensions/swe_distiller/src/extraction/checks.rs:40-62
Execution Flow
The LLM pipeline follows a specific fallback sequence, starting with deterministic heuristics before attempting LLM providers.
Provider Orchestration
The scanner supports multiple LLM backends, configured via environment variables. It attempts providers in order, falling back if a provider fails or produces an unsafe result.
| Variable | Description | Default |
|---|---|---|
SWE_DISTILLER_LLM_PROVIDERS | Comma-separated list of models/APIs | gemma4:31b-cloud,reader-lm:1.5b,jina |
SWE_DISTILLER_LLM_TIMEOUT_MS | Max time for LLM response | 60,000 |
OLLAMA_BASE_URL | Local Ollama endpoint | http://127.0.0.1:11434 |
Sources: extensions/swe_distiller/src/extraction/llm.rs:10-14, extensions/swe_distiller/src/extraction/llm.rs:70-90
Implementation Details
Metadata Extraction
The scanner utilizes a comprehensive metadata suite to maintain context during improvement passes. It extracts titles, authors, and dates from HTML meta tags, Schema.org (JSON-LD), and even byline patterns in markdown.
Sources: extensions/swe_distiller/src/metadata.rs:20-80
Removal and Sanitization
Before analysis, code and documentation are "distilled" to remove clutter:
- Content Patterns: Removal of newsletter signups, "read time" indicators, and boilerplate copyright lines.
- Metadata Blocks: Stripping of author/date blocks that are adjacent to the main H1 heading to prevent them from being treated as functional code/content.
Sources: extensions/swe_distiller/src/removal/content_patterns.rs:30-45, extensions/swe_distiller/src/lib.rs:165-175
Conclusion
The code-improvement-scanner serves as a robust automated auditor within swe-term. By combining rigid AST pattern matching with flexible but safety-gated LLM analysis, it ensures that code improvements are both structurally sound and contextually relevant, minimizing the "blast radius" of automated changes.