Language Runtimes & Hosts
Relevant source files
The following files were used as context for generating this wiki page:
Language Runtimes, also known as Language Hosts, are essential components in the Pulumi ecosystem. They act as plugins that enable the Pulumi engine to execute infrastructure as code programs written in specific programming languages such as Go, Python, Node.js, and .NET. Each supported language has a corresponding language host executable (e.g., pulumi-language-python) that implements a standardized gRPC interface.
This system allows the core Pulumi engine to remain language-agnostic while providing a consistent user experience across different languages. The language host is responsible for a variety of tasks, including running user programs, installing dependencies, and generating language-specific SDKs from provider schemas. The interaction between the engine and the language host is defined by the LanguageRuntime service in the language.proto file.
Architecture
The Pulumi engine initiates a language host as a separate process based on the runtime specified in the Pulumi.yaml project file. All communication between the engine and the language host occurs over a gRPC connection.
The diagram below illustrates the high-level interaction between the Pulumi Engine, the Language Host, and the user's program.
- The Pulumi Engine (
pulumiCLI) reads thePulumi.yamlfile to determine the project's language runtime. - It spawns the corresponding language host executable (e.g.,
pulumi-language-nodejs) as a child process. - The engine and language host establish a gRPC connection. The initial
Handshakecall passes the engine's gRPC address to the host. - The engine sends a
Runrequest to the language host. - The language host executes the user's Pulumi program (e.g.,
index.ts). - The user's program, using the language-specific Pulumi SDK, makes calls to the Resource Monitor (part of the engine) to register resources.
- The language host reports the program's completion status or errors back to the engine.
The LanguageRuntime gRPC Service
The LanguageRuntime service defines the standard contract that all language hosts must implement. This ensures that the Pulumi engine can interact with any language in a uniform way.
Sources: proto/pulumi/language.proto:16-120
Service Definition
The following sequence diagram shows a simplified pulumi up flow, highlighting the key RPC calls between the Engine and the Language Host.
This diagram illustrates the gRPC communication sequence for running a Pulumi program.
Sources: proto/pulumi/language.proto:16-120
Key RPC Methods
The LanguageRuntime service includes several critical methods for managing the lifecycle of a Pulumi program.
| Method | Description |
|---|---|
Handshake | Establishes the initial connection and protocol configuration between the engine and the language host. |
GetRequiredPackages | Computes the complete set of anticipated provider packages required by a program. This allows the engine to pre-install plugins. |
InstallDependencies | Installs language-specific dependencies for the project (e.g., npm install, pip install). It streams stdout/stderr back to the engine. |
Run | Executes a Pulumi program. The request includes project details, configuration, and the monitor address. It returns an error status upon completion. |
GeneratePackage | Generates a language-specific SDK for a given Pulumi package schema. This is the core of "sdkgen". |
GenerateProject | Generates a complete, runnable project (including scaffolding like package.json) from a PCL program. This powers pulumi convert. |
Pack | Packages a generated SDK into a language-specific artifact, such as a JAR for Java or a .tgz for Node.js. |
Cancel | Signals the language runtime to gracefully shut down and abort any ongoing operations. |
Sources: proto/pulumi/language.proto:16-120
SDK and Program Generation
A primary responsibility of a language host is code generation, which falls into two main categories: SDK generation for providers and program generation for pulumi convert.
SDK Generation (GeneratePackage)
Language hosts implement the GeneratePackage RPC to create language-specific SDKs from a provider's Pulumi schema. This process is fundamental to the Pulumi ecosystem, as it makes provider resources available in every supported language.
The general flow for SDK generation is as follows:
This diagram shows the process of generating a language-specific SDK from a Pulumi schema.
Sources: pkg/codegen/go/gen.go:1219-1277, pkg/codegen/python/gen.go:1072-1135, pkg/codegen/dotnet/gen.go:1603-1634, proto/pulumi/language.proto:106-109
The GeneratePackage function in each language's codegen module orchestrates this. It takes the schema and generates a map of filenames to file content.
- Go:
GeneratePackageinpkg/codegen/go/gen.gocreates apkgContextto manage module generation, handling Go-specific package structures and naming conventions. - Python:
GeneratePackageinpkg/codegen/python/gen.gohandles Python module structure,setup.pyorpyproject.tomlcreation, and dependency management. - .NET:
GeneratePackageinpkg/codegen/dotnet/gen.gogenerates C# projects (.csproj), namespaces, and classes for resources and inputs.
Each implementation uses language-specific templates and logic to produce idiomatic code. For example, the .NET generator creates Input and Output types, while the Python generator creates Input classes and TypedDicts.
Sources: pkg/codegen/dotnet/gen.go:1603-1634, pkg/codegen/python/gen.go:1072-1135, pkg/codegen/go/gen.go:1219-1277
Program and Project Generation
The GenerateProgram and GenerateProject RPCs are used to convert Pulumi Intermediate Language (PCL) into a target language. This is the technology behind the pulumi convert command.
GenerateProgram: Produces only the source code for the program.GenerateProject: Produces the source code plus all necessary project scaffolding, such aspackage.jsonfor Node.js orgo.modfor Go, making the output a runnable project.
This functionality allows users to migrate their infrastructure code between languages supported by Pulumi.
Sources: proto/pulumi/language.proto:92-104
Language Conformance Tests
To ensure that all language runtimes adhere to the LanguageRuntime service specification and behave consistently, Pulumi uses a suite of language conformance tests. These tests are defined in a language-agnostic way using PCL and a set of assertions.
The test process for each language is:
- Generate an SDK from a test provider schema.
- Convert the PCL test program into the target language.
- Execute the generated program using the language host.
- Verify that the resulting state and outputs match the assertions defined for the test.
This rigorous testing ensures a uniform and reliable experience for developers, regardless of their chosen language.
Sources: cmd/pulumi-test-language/README.md