Evidence Packs & Artifacts
Relevant source files
The following files were used as context for generating this wiki page:
Evidence Packs and Artifacts represent the persistence layer of scoutline research. They store the results of web searches, page reads, site crawls, and repository explorations to ensure work is reproducible and quotable.
The system captures raw data from providers, redacts sensitive credentials, applies output budgets if requested, and maintains a linked log for historical recovery. Users interact with these artifacts through the --save flag and the history command suite.
Artifact Storage and Identification
The system generates a unique requestId for every captured artifact to ensure lexicographical ordering and collision resistance.
Request ID Generation
The newRequestId function creates IDs using a compact UTC timestamp followed by a four-character lowercase hex tail. This format supports time-based sorting and uniquely identifies individual invocations even when multiple occur within the same second.
Sources: packages/scoutline/tests/artifacts-store.test.js:25-50
Storage Location
The storage directory resolution follows a strict hierarchy. The SCOUTLINE_ARTIFACTS_DIR environment variable takes highest precedence. If not set, the system defaults to the artifacts subdirectory within the scoutline configuration root.
Sources: packages/scoutline/tests/artifacts-store.test.js:82-95
Security and Permissions
Storage operations enforce high-security file permissions. Directories are created with 0700 (owner read/write/execute) and files with 0600 (owner read/write).
Sources: packages/scoutline/tests/artifacts-store.test.js:120-125
The diagram shows how storage paths are resolved and how unique IDs are assigned before writing to disk.
Master Artifacts and Export Copies
The system distinguishes between internal "master" artifacts and user-facing "export" copies.
Master Artifact Structure
A master artifact is a JSON or Markdown file stored in the artifacts directory. It contains exactly three root keys: schemaVersion, requestId, and result. The result field holds the exact data serialized to standard output.
Sources: packages/scoutline/tests/save-artifact.test.js:108-115
Exporting Evidence
Users use the --save [path] flag to create an export copy. The system performs an atomic check-and-place operation. It refuses to overwrite existing files unless the --save-force flag is present. Relative export paths resolve against the current working directory at save time but are stored as absolute paths in the history log.
Sources: packages/scoutline/tests/save-artifact.test.js:46-55, packages/scoutline/tests/save-artifact.test.js:335-345
Artifact Formats
| Format | Extension | Description |
|---|---|---|
| JSON | .json | Raw data payload for machine consumption. |
| Markdown | .md | Human-readable report with a comment header containing metadata. |
| Sources: packages/scoutline/tests/artifacts-store.test.js:112-118, packages/scoutline/tests/save-artifact.test.js:275-285 |
Output Budgeting and Compaction
When the --max-chars flag is used, scoutline applies a "ladder" priority logic to fit results within a character limit while preserving critical evidence in the stored artifact.
Compaction Logic
Compaction shrinks the data sent to standard output but preserves the full, untrimmed result in the master artifact. This allows users to review the complete data later via history show.
Sources: packages/scoutline/tests/output-budget-t4.test.js:410-425
Truncation and Omission Markers
The system uses a backward-scan algorithm to bleed text bodies before dropping entire sections. Trimmed fields are prefixed with an omission marker (…) and carry a truncated: true flag.
Sources: packages/scoutline/tests/output-budget-t4.test.js:175-200, packages/scoutline/tests/output-budget-t4.test.js:630-645
Floor Envelopes
Every command defines a "floor" envelope. This is the minimum set of metadata (like URLs and query terms) that the system will never destroy, even if the result exceeds the requested budget.
Sources: packages/scoutline/tests/output-budget-t5.test.js:325-335
History and Dossier Rendering
The history export command generates a comprehensive markdown dossier from the stored research journal.
Dossier Structure
The dossier organizes findings newest-first. Each section represents a unique research finding and includes:
- Metadata: Query, capability used, provider (including cache status), and timestamp.
- Provenance: A line per result row containing the URL, timestamp, and a
contentHashfor integrity verification. - Inlined Content: If a master artifact exists for a finding, its body is inlined directly within a JSON code fence.
Sources: packages/scoutline/tests/history-export.test.js:125-155, packages/scoutline/tests/history-export.test.js:485-500
Record Redaction
Artifacts and logs automatically redact sensitive information. The system scans result content for configured environment secrets and replaces them with a [REDACTED] marker before writing to disk.
Sources: packages/scoutline/tests/save-artifact.test.js:155-175, packages/scoutline/tests/output-budget-t4.test.js:460-475
The sequence diagram illustrates the dossier generation process, highlighting the local read-only nature of the operation.
Summary
Evidence Packs and Artifacts provide a robust, secure, and reproducible audit trail for scoutline research. By combining atomic storage, unique request IDs, and flexible dossier exports, the system enables users to preserve findings while maintaining high standards for data integrity and credential safety. Artifacts ensure that even when results are budgeted for immediate consumption, the full scope of evidence remains available for later investigation.