Usage Ledger & Output Budget
Relevant source files
The following files were used as context for generating this wiki page:
The Usage Ledger and Output Budget systems manage local resource tracking and data presentation constraints. The Usage Ledger records billable call attempts across providers and capabilities, storing these counters in a local JSON file for historical reporting. The Output Budget system applies size constraints to command results, using "ladder" rules to selectively trim or drop content to fit within a specified character limit.
Sources: packages/scoutline/src/commands/usage.ts:1-25, packages/scoutline/src/lib/output-budget.ts:1-15
Usage Ledger Architecture
The Usage Ledger system uses a local JSON file (usage.json) to track billable call attempts. It operates without credentials and records only counters, never storing queries, URLs, or result content.
Data Flow and Persistence
Consumption events originate from billable invokes in the execution layer. A ConsumptionSink captures these events and merges them into the ledger. The system uses an asynchronous file lock and atomic replacement to ensure data integrity during writes.
The system maintains a 90-day retention window. When a write occurs on a new UTC day, the system prunes entries older than 90 days.
Sources: packages/scoutline/src/lib/usage-ledger.ts:1-30, packages/scoutline/src/lib/consumption.ts:1-15
Ledger Data Structure
The ledger organizes data by UTC calendar dates, provider IDs, and capability IDs.
| Field | Type | Description |
|---|---|---|
attempts | number | Total billable call attempts. |
firstTries | number | Attempts where attempt === 1. |
exactUnits | number | Consumed units from exact providers. |
estimateUnits | number | Estimated units consumed. |
unknownCount | number | Count of events with unknown amounts. |
Sources: packages/scoutline/src/lib/usage-ledger.ts:40-60, packages/scoutline/src/commands/usage.ts:40-55
Output Budget System
The Output Budget system fits large data payloads into character-constrained environments. It applies a series of "Ladder Rules" that transform an envelope until it meets the maxChars budget or reaches a "floor" state.
Budget Execution Logic
The system measures the JSON-serialized size of the payload. If the size exceeds the budget, it iteratively applies rules from a prioritized list (the ladder).
Ladder Rules
Rules are command-specific and categorized into two primary types:
- Trim Rules: Reduce the size of string fields (e.g., halving the length of a summary) and prepend an omission marker (
…). - Drop Rules: Remove optional fields or entire items from arrays (e.g., dropping the lowest-ranked search result).
The engine ensures that "never-cut" fields, such as URLs and titles, remain intact across all shrink levels.
Sources: packages/scoutline/src/lib/output-budget.ts:30-80, packages/scoutline/src/index.ts:1200-1210
Budget Persistence
When a budget is applied, the system saves the full, unbudgeted result to the artifacts store. The budgeted output includes a compaction object containing metadata about the original result.
Compaction Metadata
The compaction object is injected into the data payload and contains two fields:
budget: The character limit requested by the user.ref: A unique reference ID (timestamp + random suffix) pointing to the master artifact.
Sources: packages/scoutline/src/lib/output-budget-persistence.ts:10-30, packages/scoutline/src/index.ts:1230-1250
Artifact Store Integration
The system redacts configured secrets from the original payload before saving it to disk. Users can recover the full untrimmed content using the scoutline history show <requestId> command.
| Action | Component | Description |
|---|---|---|
| Redaction | redactSecrets | Removes API keys from master artifacts. |
| Storage | atomicPlaceNoClobber | Writes master file to artifacts directory. |
| Linking | saveRef | Cross-links journal entries and budget artifacts. |
Sources: packages/scoutline/src/index.ts:1225-1260, packages/scoutline/src/lib/output-budget-persistence.ts:40-60
Usage Reporting
The usage command aggregates ledger data into human-readable tables or raw JSON envelopes.
Aggregation Logic
Users can specify a window in days (default 7). The aggregator sums counters across the window for each provider and capability. Providers and capabilities are always emitted in ascending lexicographical order.
Presentation Modes
- Data Mode: Returns a
UsageReportJSON envelope withschemaVersion,windowDays, and provider totals. - Text Modes (TTY/Markdown): Renders a fixed-order table with right-aligned numeric columns for attempts and units.
Sources: packages/scoutline/src/commands/usage.ts:70-120, packages/scoutline/src/commands/usage.ts:160-200
Summary
The Usage Ledger provides transparent local accounting of API consumption, supporting cost monitoring without requiring network access. Complementing this, the Output Budget system allows users to constrain output sizes while preserving critical metadata and ensuring that full evidentiary results are always recoverable via the local artifact history. Together, these systems provide control over data volume and resource transparency within the Scoutline environment.