Science Suppliers
Relevant source files
The following files were used as context for generating this wiki page:
Science Suppliers are keyless scholarly APIs that provide Scoutline with specialized scholarly research capabilities. They enable the Science Capability, allowing users to search for and retrieve scholarly records known as ScienceWork. This system is distinct from standard AI providers as it operates without required credentials and utilizes a specific fan-out and deduplication logic tailored for academic data.
Sources: CONTEXT.md:310-316, CONTEXT.md:395-400
Architecture and Registry
The Science Supplier system integrates five primary scholarly indices. These suppliers are utilized by the scoutline science search and scoutline science get commands.
Registered Suppliers
The system maintains a fixed registry of scholarly APIs:
- arXiv: A repository of electronic preprints.
- OpenAlex: A global index of scholarly entities.
- Crossref: A digital object identifier (DOI) registration agency.
- PubMed: A database of biomedical literature.
- Europe PMC: An open-access repository for biomedical and life sciences research.
Sources: packages/scoutline/tests/science-docs.test.js:63, CONTEXT.md:310-312
Execution Priority
When performing a search without a specific provider pin, Scoutline executes a fan-out request following a specific internal priority order:
- OpenAlex
- arXiv
- Crossref
- PubMed
- Europe PMC
Sources: packages/scoutline/tests/science-fanout-merge.test.js:41-45, packages/scoutline/tests/science-command.test.js:31-36
Data Operations
The Science Supplier module supports two primary verbs: search and get.
Science Search
The search command performs a parallel fan-out across all enabled science suppliers. It accepts multiple positionals which are joined with spaces into a single query string.
- Fan-out: Requests are sent to all available suppliers simultaneously.
- Deduplication: Results are merged based on DOI (Digital Object Identifier) first, with a fallback to normalized URLs.
- Union Enrichment: Merged records combine fields from multiple suppliers; if conflicts exist, the highest-priority supplier's data is preserved.
Sources: packages/scoutline/tests/science-fanout-merge.test.js:14-23, packages/scoutline/tests/science-command.test.js:231-240
Science Get
The get command retrieves a specific ScienceWork using a persistent identifier. The system routes the request to suppliers based on the identifier type:
- DOI: Routed to all suppliers except arXiv.
- PMID: Routed to OpenAlex, Europe PMC, and PubMed.
- arXiv ID: Routed exclusively to arXiv.
Sources: packages/scoutline/tests/science-command.test.js:38-41, packages/scoutline/tests/science-command.test.js:283-294
Data Flow Diagram
The diagram represents how requests are dispatched to various scholarly suppliers and subsequently merged or routed based on the command type.
Sources: packages/scoutline/tests/science-fanout-merge.test.js:14-25, packages/scoutline/tests/science-command.test.js:38-41
Constraints and Configuration
Command Controls
Science commands support specific metadata filters. If a pinned supplier does not support a requested control, the command fails with UNSUPPORTED_OPTION.
| Control | Accepted Format | Description |
|---|---|---|
--author | String | Filters by author name. |
--year | YYYY or YYYY:YYYY | Filters by single year or closed range. |
--venue | String | Filters by publication venue (e.g., Nature). |
--type | Union Vocabulary | Filters by document type (article, preprint, review, etc). |
Sources: packages/scoutline/tests/science-command.test.js:42-49, packages/scoutline/tests/science-command.test.js:52-58
Credentials and Quota
Science Suppliers are generally keyless, but certain suppliers support optional keys for higher rate-limit tiers.
- Environment Variables:
OPENALEX_API_KEY,NCBI_API_KEY. - Quota Policy: All science suppliers are categorized as
always-unknownin the quota authority table because they do not provide spend signals.
Sources: packages/scoutline/tests/science-docs.test.js:31-33, packages/scoutline/tests/science-docs.test.js:161-165
Journaling and Skeletons
Every science operation appends an entry to the Scoutline Journal.
- Search Skeleton: Stores a merged list of
urlandtitlepairs for all results. - Get Skeleton: Stores a single-work identity.
- Content Hash: A SHA-256 hash of the normalized skeleton serialization.
- Provider Routing: Recorded as
{mode: "fanout", arms: [...]}for searches or{mode: "single", effective: <id>}for get requests.
Sources: packages/scoutline/tests/science-journal.test.js:13-18, packages/scoutline/tests/science-fanout-merge.test.js:27-31
Summary
Science Suppliers provide a robust scholarly vertical for Scoutline. By leveraging parallel fan-out and DOI-based deduplication, the system ensures comprehensive and accurate retrieval of academic literature without requiring complex API configurations or paid credits. Integration with the Scoutline Journal ensures that all scholarly lookups are recorded with user-visible identifiers for future recall.