API Endpoints: Ingestion
Relevant source files
The following files were used as context for generating this wiki page:The Aldeci platform's ingestion endpoints are the primary entry point for security and software development lifecycle (SDLC) artifacts. These FastAPI-based endpoints are designed to accept push-style uploads of various data types, such as Software Bills of Materials (SBOMs), static analysis results (SARIF), and CVE information. Once received, the data is validated, normalized into a standard internal format, and stored for processing by the main analysis pipeline.
This system provides a consistent interface for both automated CI/CD tools and local command-line operations to feed data into the platform. The ingestion mechanism is a critical first step that enables the subsequent correlation, risk scoring, and evidence bundling features. For more details on the full workflow, see the Pipeline Orchestration and Evidence Bundles & CI Agent documentation.
Sources: README.md:3, README.md:75, apps/api/app.py
Ingestion Architecture
The ingestion system is built around a main FastAPI application that routes incoming requests to dedicated handlers. The process is mirrored by a CLI for local execution, ensuring parity between remote and local workflows.
Sources: README.md:75, apps/api/app.py, core/cli.py
High-Level Data Flow
The following diagram illustrates the general flow of data from a client to the final storage and pipeline trigger.
This diagram shows a client uploading an artifact, which is then validated, normalized, and archived before being passed to the pipeline for analysis.
Sources: README.md:124, apps/api/app.py:253, apps/api/pipeline.py
Core Components
| Component | Description | Source File(s) |
|---|---|---|
| FastAPI App | The main web server application that defines the endpoints, enforces security (API keys), and manages request handling. | apps/api/app.py |
| InputNormalizer | A key class responsible for parsing various input formats (e.g., CycloneDX, SPDX, SARIF) and transforming them into standardized Pydantic models. | apps/api/normalizers.py |
| PipelineOrchestrator | Coordinates the analysis of normalized artifacts, evaluates guardrails, runs contextual modules, and generates the final results. | apps/api/pipeline.py |
| ArtefactArchive | Manages the storage of both raw and normalized artifacts on disk, ensuring they are saved securely within allowlisted directories. | core/storage.py |
CLI (core/cli) | Provides a command-line interface with parity to the API, allowing users to run the same ingestion and pipeline logic locally. | core/cli.py |
Sources: README.md:253-254, apps/api/normalizers.py, core/cli.py, core/storage.py
Ingestion Endpoints
The API exposes a set of POST endpoints under the /inputs/ path, each dedicated to a specific artifact type. These endpoints accept JSON data, as well as gzip or zip archives.
| Endpoint | HTTP Method | Description | Details |
|---|---|---|---|
/inputs/sbom | POST | Upload an SBOM in CycloneDX or SPDX format. | Normalizes component tokens and versions. |
/inputs/sarif | POST | Upload scanner findings in SARIF format. | Deduplicates findings by rule ID and normalizes severity labels. |
/inputs/cve | POST | Upload CVE or CISA KEV advisories. | Enriches data with staleness and EPSS scores. |
/inputs/design | POST | Upload design documents (CSV format). | The context from these files is used to correlate business impact with findings. |
Sources: README.md:360-363
After artifacts are ingested, the full analysis can be triggered by calling the /pipeline/run endpoint. This orchestrator uses the cached artifacts to build a crosswalk between design, SBOMs, and findings, evaluate guardrails, and generate a comprehensive report.
Sources: README.md:364, apps/api/pipeline.py
Data Normalization
A crucial step in the ingestion process is normalization, handled by the InputNormalizer class. This component ensures that regardless of the source tool or specific format (e.g., Syft-generated CycloneDX vs. a generic SPDX file), the data is converted into a consistent internal representation.
Sources: apps/api/normalizers.py
Normalizer Class Structure
The InputNormalizer uses Pydantic models to represent the standardized structure for different artifact types.
This diagram shows the InputNormalizer and some of the key data structures it produces, such as NormalizedSBOM and NormalizedSARIF.
Sources: apps/api/normalizers.py:160-330
A special helper function, _convert_snyk_payload_to_sarif, exists to handle proprietary Snyk JSON output by converting it into the standard SARIF format before further processing.
Sources: apps/api/normalizers.py:270-330
End-to-End Ingestion Sequence
The following sequence diagram shows the interactions between components during a typical artifact upload and processing workflow.
This diagram illustrates the two-stage process: first, artifacts are uploaded and normalized via /inputs endpoints. Second, the /pipeline/run endpoint is called to trigger analysis on the stored artifacts, which ultimately results in an evidence bundle.
Sources: README.md:253-254, apps/api/app.py, apps/api/pipeline.py, core/storage.py, backend/api/evidence/router.py
Conclusion
The ingestion endpoints form the foundation of the Aldeci platform's data processing capabilities. By providing a robust and standardized entry point for diverse security artifacts, they enable a decoupled architecture where data can be collected asynchronously and processed on-demand. This flow is essential for integrating with various CI/CD systems and providing a consistent analysis experience for both API and CLI users.
Sources: README.md:75, apps/api/app.py:253