AST to IR Transformation Flow
Relevant source files
The following files were used as context for generating this wiki page:
The AST to IR Transformation Flow in the Jazz compiler represents the multi-stage pipeline where high-level surface syntax is parsed, analyzed, and lowered into Intermediate Representations (IR) for optimization and code generation. The current active implementation path is hosted within the jazz-next/ directory, while jazz-hs/ and jazz2/ serve as historical or experimental references for LLVM-based backend logic and design stubs.
The transformation process begins with tokenization and parsing of the surface AST, followed by a lowering boundary that converts the parse-surface AST into an analyzer-facing core AST. This core AST undergoes scope-aware analysis and type inference before being prepared for the backend, which is planned to utilize an LLVM or QBE infrastructure for performance targets.
Sources: jazz-next/README.md:1-25, README.md:13-35
Frontend: Parsing and Lowering
The frontend transformation starts with the lexer and parser. The jazz-next implementation utilizes a minimal tokenization strategy to bootstrap the parser, which handles the surface syntax including dot-terminated source forms and brace-bodied module declarations.
Lowering Boundary
A critical stage in the flow is the transition from the Parser.AST to the Compiler.AST. This "lowering boundary" separates the structures required for syntactic representation from the semantic structures required by the analyzer and type system.
The diagram above shows the linear progression from raw source code to the core AST used for semantic analysis.
| Component | File Path | Description |
|---|---|---|
| Lexer | src/JazzNext/Compiler/Parser/Lexer.hs | Performs minimal tokenization for parser bootstrap. |
| Parser | src/JazzNext/Compiler/Parser.hs | Handles surface syntax slices. |
| Parse AST | src/JazzNext/Compiler/Parser/AST.hs | Defines the contract for the parse-surface AST. |
| Lowering | src/JazzNext/Compiler/Parser/Lower.hs | Maps parse-AST nodes to analyzer-AST nodes. |
Sources: jazz-next/README.md:16-23, README.md:20-22
Analysis and Semantic Verification
Once the AST is lowered to the core representation (Compiler.AST), it enters the analysis phase. This stage is responsible for ensuring the program is semantically valid before any attempt is made to generate IR.
Analyzer Logic
The Analyzer.hs module performs several key checks:
- Scope-aware rebinding: Detects issues like
W0001(rebinding warnings). - Coherence checks: Verifies signature adjacency and binding/signature consistency.
- Diagnostic Generation: Produces deterministic warnings and errors via
Diagnostics.hs.
Type Inference
The TypeInference.hs module processes the analyzed AST to perform strong, static typing. It carries the results of the analysis, including warnings and semantic errors, forward into the final compile results handled by the Driver.
Sources: jazz-next/README.md:24-30, README.md:13-15
Backend: IR Generation and JIT
The transformation from the core AST to a backend-specific IR (like LLVM IR) is supported by legacy and experimental modules in jazz-hs and planned for full integration in jazz-next.
LLVM IR Construction
The project uses llvm-hs-pure and llvm-hs to interface with the LLVM toolchain. The transformation involves mapping Jazz's functional constructs (lambdas, curried functions, and immutable bindings) to LLVM's instruction-based representation.
The sequence diagram illustrates how the core AST is converted into an LLVM Module and subsequently handled by a Just-In-Time (JIT) compiler.
LLVM Components
The FFI layer in jazz-hs provides the necessary C++ hooks for advanced IR manipulation:
- Module Management:
LLVM_Hs_cloneAsThreadSafeModuleis used to prepare modules for the OrcJIT. - Metadata: Extensive support for DWARF debug metadata (e.g.,
DILocation,DISubprogram) is included to track source-level information in the generated IR. - IRBuilder: A monadic API for building LLVM modules, which includes automated name generation and block label management.
Sources: jazz-hs/local-deps/llvm-hs/src/LLVM/Internal/FFI/OrcJITC.cpp:92-105, jazz-hs/local-deps/llvm-hs-pure/CHANGELOG.md:118-125, jazz-hs/local-deps/llvm-hs-pretty/README.md:65-115
Transformation Mapping
The following table describes the mapping between high-level Jazz constructs and their intended or experimental IR representations.
| Jazz Construct | Transformation Logic | Target IR / Representation |
|---|---|---|
| First-class Functions | Lambdas are lowered to global functions. | AST.GlobalDefinition |
| Partial Application | Functions are curried by default. | Closure or wrapper functions. |
| Immutable Bindings | Direct assignment. | LLVM Constants or local references. |
| Impure Functions | Denoted with !. | Purity enforcement in Analyzer. |
Sources: README.md:16-19, README.md:92-105, jazz-hs/local-deps/llvm-hs-pretty/README.md:78-95
Summary
The AST to IR transformation flow in Jazz moves from a flexible surface syntax through a rigorous lowering and analysis phase in jazz-next. The transition to IR leverages powerful backends like LLVM, utilizing a structured mapping of functional concepts to a lower-level, instruction-based execution model. This flow ensures that language features like currying and strong typing are preserved and optimized during the compilation process.