Compiler Pipeline & IR
Relevant source files
The following files were used as context for generating this wiki page:
The Crucible compiler pipeline transforms service models defined in the .cru language into a shared Intermediate Representation (IR). This IR provides a settled, language-independent definition of model semantics, including nullability, constraints, HTTP bindings, and pagination rules. The system ensures that all generators—whether producing servers, clients, or documentation—work from a single source of truth to maintain consistency across different target languages.
Sources: AGENTS.md, crates/crucible-cli/src/main.rs:432-440
Compiler Architecture & Data Flow
The pipeline consists of a front-end that parses the Crucible modelling language and a back-end that consumes the resulting IR to generate code. The compiler reads manifests (crucible.toml) and model sources, performs name resolution, and executes validity checks before lowering the abstract syntax tree to the IR.
Compilation Phases
- Loading: The system loads a Crucible package from a directory, identifying the namespace and source files.
- Parsing & Resolution: A lossless lexer and recovery-capable parser create a typed syntax tree; the compiler then resolves names and imports.
- Lowering: The frontend lowers the resolved model to the IR structures defined in
crucible-ir. - Validation & Guarding: The pipeline runs compatibility guards against frozen baselines to prevent breaking changes and executes project policy rules.
- Generation: Target-specific generators (e.g., Rust, TypeScript, Java) consume the IR to produce SDKs or documentation.
Sources: AGENTS.md, crates/crucible-cli/src/main.rs:823-863, crates/crucible-lang/tests/package.rs:200-220
The diagram shows the sequence from source files through the frontend compiler phases to the shared IR and subsequent generation. Sources: AGENTS.md, crates/crucible-cli/src/main.rs:835-850
Intermediate Representation (IR) Structure
The crucible-ir crate owns the IR, which is serializable as a release snapshot format. This format is stable; renaming fields or adding non-default fields in the IR source code breaks existing committed releases. The IR settles model semantics once for all downstream consumers.
Key IR Components
| Component | Description |
|---|---|
Service | The root container defining the API version, protocol, authentication, and operations. |
Shape | Defines data types: structures, unions, enums, lists, maps, and simple scalars. |
Operation | Describes a single API call, its input/output shapes, and HTTP bindings. |
Traits | Metadata and behavior modifiers applied to shapes or operations (e.g., @internal, @deprecated). |
Permissions | The registry of resource types and associated permission values. |
Sources: AGENTS.md, crates/crucible-cli/src/main.rs:1150-1175, crates/crucible-codegen-docs/src/markdown.rs:200-250
Versioning & Compatibility Pipeline
The compiler pipeline manages service evolution through "SDK lines" and "frozen releases." A build fails if a breaking change is detected against the package's baseline unless a migration is explicitly declared.
Release Management Process
- Freeze: Captures a snapshot of the compiled model into
releases/<version>.json. - SDK Lines: Generates code for a specific supported release by projecting the current model as that release sees it.
- Audience Projection: Public outputs omit
@internalAPI components unless the internal audience is explicitly selected.
Sources: crates/crucible-cli/src/main.rs:115-135, crates/crucible-cli/src/main.rs:770-785, AGENTS.md
This sequence diagram illustrates the integration of the compatibility guard within the standard compilation flow. Sources: crates/crucible-cli/src/main.rs:830-865
Metadata & Documentation Generation
The pipeline integrates documentation generation directly into the compiler workflow. The crucible-codegen-common and crucible-codegen-docs modules provide shared logic for escaping text, redacting @sensitive data, and rendering Markdown.
Documentation Components
- README.md: Provides installation instructions, client construction, and usage snippets for human developers.
- AGENTS.md: Contains terse instructions and exact identifier names optimized for AI coding agents.
- API Reference: Generates mdBook-compatible Markdown pages detailing operations, data types, and permissions.
Sources: crates/crucible-codegen-common/src/guide.rs:1-25, crates/crucible-codegen-docs/src/markdown.rs:1-30
Redaction Rules
The compiler enforces specific rules for sensitive information:
- Members marked
@sensitiveare excluded from logs and string representations. - Generators must use common redaction markers (e.g.,
REDACTED) defined incrucible-codegen-common.
Sources: AGENTS.md, crates/crucible-codegen-common/src/guide.rs:30-45
Conclusion
The Compiler Pipeline & IR system serves as the foundational orchestrator for the Crucible project. By enforcing a single-pass lowering to a stable IR, the system guarantees that complex model semantics—such as pagination tokens and idempotency rules—behave identically across all supported programming languages and infrastructure providers.