Scoutline is an agent-first Command Line Interface (CLI) for web research, multi-modal source investigation, and repository exploration. The system architecture prioritizes reproducible results, integrated caching, and clear provenance of data. It manages provider routing and output budgeting to ensure that agents receive information within defined character limits while preserving essential metadata.

Sources: packages/scoutline/tests/agent-registration-docs.test.js:35-43, packages/scoutline/package.json:2-4

Local Context System

The Local Context system extracts steering information from local files or standard input to guide research queries. It identifies document structure to derive terms, sub-queries, and research biases.

Context Parsing Logic

The parseContextText function processes raw text to extract specific information types:

ATX Headings: Extracts levels 1–6 in document order.
Questions: Identifies lines ending in ? (limited to 200 characters).
Terms: Tokenizes headings and questions, filters out stopwords, and applies length bounds (4–40 characters).
Sub-Queries: Interleaves headings and questions in document order, capping at MAX_SUBQUERIES (8).

Sources: packages/scoutline/tests/context-file.test.js:54-200

Privacy and Transmission Boundaries

The system maintains strict boundaries regarding what data leaves the local machine. When using --context-mode, the system communicates different levels of data:

Organize: Default mode; results stay local.
Bias/Both: Changes the cache key and transmits derived terms to providers.
Metadata: The research envelope includes context fields that only contain derived counts or SHA-256 hashes, rather than raw text.

Sources: packages/scoutline/tests/context-docs.test.js:31-40, packages/scoutline/tests/context-docs.test.js:65-72

Generating architecture diagram...

The diagram above illustrates the transformation of local context from raw input into structured steering data.

Output Budgeting and Compaction (ADR-0007)

Scoutline uses a "Ladder" system to enforce character limits on research outputs. This ensures that large payloads are truncated gracefully for agents while preserving critical provenance.

Compaction Ladders

Different commands use specialized ladders to determine priority during truncation:

READ_LADDER: Preserves URLs, titles, and headings. Trims later paragraphs first and drops bottom sections last.
READ_EXTRACT_LADDER: Trims field values but never drops field names or URL values.
CRAWL_LADDER: Never cuts page URLs; trims content first and drops trailing pages late.
RESEARCH_LADDER: Ensures source citations survive the longest while report sections are trimmed.

Sources: packages/scoutline/tests/output-budget-t4.test.js:140-155, packages/scoutline/tests/output-budget-t4.test.js:233-245

Budget Constraints

Constant	Value	Description
`MAX_CONTEXT_BYTES`	262144	Maximum size allowed for a context source file.
`MAX_SUBQUERIES`	8	Cap for interleaved sub-queries derived from context.
`MAX_TERMS`	12	Maximum number of terms preserved in the term pipeline.
`COMPACTION_STAMP_RESERVE`	Internal	Character reserve for the compaction metadata stamp.

Sources: packages/scoutline/tests/context-file.test.js:273-277, packages/scoutline/tests/context-file.test.js:203-210

Artifact Storage and Recovery

The system stores full, untrimmed response envelopes as artifacts, allowing users to recover original data even after output compaction.

Atomic Discipline

Storage operations follow a strict atomic discipline to prevent data corruption:

Directory Permissions: Uses 0700 for artifact directories.
File Permissions: Uses 0600 for individual artifact files.
RequestId: Generated as <UTC compact>-<4 lowercase hex> (e.g., 20260829T142233Z-7f3a).
Overwrite Protection: The writeArtifact function refuses to overwrite existing targets unless the force option is provided.

Sources: packages/scoutline/tests/artifacts-store.test.js:12-25, packages/scoutline/tests/artifacts-store.test.js:111-125

Generating architecture diagram...

This sequence shows how the system ensures data persistence before presenting compacted information to the user.

Provider and Capability Architecture

Scoutline routes requests to specialized providers through defined capabilities.

Scholarly (Science) Capability

The science module enablesscholarly lookups across five primary suppliers: arXiv, OpenAlex, Crossref, PubMed, and Europe PMC. It supports keyless scholarly index access with optional upgrade tiers for specific API keys.

Command	Capability ID	Description
`scoutline science search`	`science.search`	Search for scholarly publications.
`scoutline science get`	`science.get`	Retrieve details for a specific identifier.

Sources: packages/scoutline/tests/science-docs.test.js:103-115, packages/scoutline/tests/science-docs.test.js:22-30

Quota and Policy Management

Providers are mapped to authority policies. Science suppliers are currently mapped to always-unknown because they operate as keyless scholarly indices without spend signals. This excludes them from availability ranking in initial versions.

Sources: packages/scoutline/tests/science-docs.test.js:144-155

Agent Registration

The project includes a mechanism for registering the CLI with agent tools via a stamp file.

Stamp File: agent-registration.json stored under the config root.
Backup Mechanic: Uses .scoutline-bak as a disaster-recovery-only escape hatch. The system does not use backups as an uninstallation mechanism.
Skill Deployment: Marketplace skills are shipped in the npm package under packages/scoutline/skills/.

Sources: packages/scoutline/tests/agent-registration-docs.test.js:16-25, CONTRIBUTING.md:10-15

Conclusion

Scoutline's architecture is centered on providing a reliable, character-limited interface for automated agents. By combining a robust local context parser with an atomic artifact store and prioritized compaction ladders, the system ensures that critical metadata and provenance are preserved while maintaining high performance across multiple search and scholarly providers.

Sources: packages/scoutline/tests/agent-registration-docs.test.js:35-43, packages/scoutline/tests/output-budget-t4.test.js:14-25

Resources

AI code review
Bug Bench
AI coding statistics
Coding standards
Agents
Wikis

View on GitHub

Back to directory

Repository

vikasagarwal101/scoutline

Overview

Introduction & Quick Start Installation & Setup

System Architecture

Architecture Concepts & Context CLI Dispatch & Command Invocation

Core Features

Provider Integration

Provider Selection & Routing Z.AI & MiniMax AI Integrations Search Engine Integrations

Data Management & Flow

Local Caching System Research Journal & History Evidence Packs & Artifacts Usage Ledger & Output Budget

Extensibility and Customization

MCP Tools Integration Code Mode for Agent Automation Agent Registration

Operations & Troubleshooting

Configuration Management Diagnostics & Quota Checking Error Handling & Fallback Mechanisms Testing Strategies

Architecture Concepts & Context

Relevant source files

The following files were used as context for generating this wiki page:

packages/scoutline/tests/science-docs.test.js
packages/scoutline/tests/agent-registration-docs.test.js
packages/scoutline/tests/output-budget-t4.test.js
packages/scoutline/tests/context-file.test.js
packages/scoutline/tests/artifacts-store.test.js
packages/scoutline/tests/context-docs.test.js
scoutline-augmentation-spec.md

Sources: packages/scoutline/tests/agent-registration-docs.test.js:35-43, packages/scoutline/package.json:2-4

Local Context System

Context Parsing Logic

The parseContextText function processes raw text to extract specific information types:

ATX Headings: Extracts levels 1–6 in document order.
Questions: Identifies lines ending in ? (limited to 200 characters).
Terms: Tokenizes headings and questions, filters out stopwords, and applies length bounds (4–40 characters).
Sub-Queries: Interleaves headings and questions in document order, capping at MAX_SUBQUERIES (8).

Sources: packages/scoutline/tests/context-file.test.js:54-200

Privacy and Transmission Boundaries

The system maintains strict boundaries regarding what data leaves the local machine. When using --context-mode, the system communicates different levels of data:

Organize: Default mode; results stay local.
Bias/Both: Changes the cache key and transmits derived terms to providers.
Metadata: The research envelope includes context fields that only contain derived counts or SHA-256 hashes, rather than raw text.

Sources: packages/scoutline/tests/context-docs.test.js:31-40, packages/scoutline/tests/context-docs.test.js:65-72

Generating architecture diagram...

The diagram above illustrates the transformation of local context from raw input into structured steering data.

Output Budgeting and Compaction (ADR-0007)

Scoutline uses a "Ladder" system to enforce character limits on research outputs. This ensures that large payloads are truncated gracefully for agents while preserving critical provenance.

Compaction Ladders

Different commands use specialized ladders to determine priority during truncation:

READ_LADDER: Preserves URLs, titles, and headings. Trims later paragraphs first and drops bottom sections last.
READ_EXTRACT_LADDER: Trims field values but never drops field names or URL values.
CRAWL_LADDER: Never cuts page URLs; trims content first and drops trailing pages late.
RESEARCH_LADDER: Ensures source citations survive the longest while report sections are trimmed.

Sources: packages/scoutline/tests/output-budget-t4.test.js:140-155, packages/scoutline/tests/output-budget-t4.test.js:233-245

Budget Constraints

Constant	Value	Description
`MAX_CONTEXT_BYTES`	262144	Maximum size allowed for a context source file.
`MAX_SUBQUERIES`	8	Cap for interleaved sub-queries derived from context.
`MAX_TERMS`	12	Maximum number of terms preserved in the term pipeline.
`COMPACTION_STAMP_RESERVE`	Internal	Character reserve for the compaction metadata stamp.

Sources: packages/scoutline/tests/context-file.test.js:273-277, packages/scoutline/tests/context-file.test.js:203-210

Artifact Storage and Recovery

The system stores full, untrimmed response envelopes as artifacts, allowing users to recover original data even after output compaction.

Atomic Discipline

Storage operations follow a strict atomic discipline to prevent data corruption:

Directory Permissions: Uses 0700 for artifact directories.
File Permissions: Uses 0600 for individual artifact files.
RequestId: Generated as <UTC compact>-<4 lowercase hex> (e.g., 20260829T142233Z-7f3a).
Overwrite Protection: The writeArtifact function refuses to overwrite existing targets unless the force option is provided.

Sources: packages/scoutline/tests/artifacts-store.test.js:12-25, packages/scoutline/tests/artifacts-store.test.js:111-125

Generating architecture diagram...

This sequence shows how the system ensures data persistence before presenting compacted information to the user.

Provider and Capability Architecture

Scoutline routes requests to specialized providers through defined capabilities.

Scholarly (Science) Capability

Command	Capability ID	Description
`scoutline science search`	`science.search`	Search for scholarly publications.
`scoutline science get`	`science.get`	Retrieve details for a specific identifier.

Sources: packages/scoutline/tests/science-docs.test.js:103-115, packages/scoutline/tests/science-docs.test.js:22-30

Quota and Policy Management

Sources: packages/scoutline/tests/science-docs.test.js:144-155

Agent Registration

The project includes a mechanism for registering the CLI with agent tools via a stamp file.

Stamp File: agent-registration.json stored under the config root.
Backup Mechanic: Uses .scoutline-bak as a disaster-recovery-only escape hatch. The system does not use backups as an uninstallation mechanism.
Skill Deployment: Marketplace skills are shipped in the npm package under packages/scoutline/skills/.

Sources: packages/scoutline/tests/agent-registration-docs.test.js:16-25, CONTRIBUTING.md:10-15

Conclusion

Sources: packages/scoutline/tests/agent-registration-docs.test.js:35-43, packages/scoutline/tests/output-budget-t4.test.js:14-25

Resources

AI code review
Bug Bench
AI coding statistics
Coding standards
Agents
Wikis

View on GitHub