/Alonso-pixel/pulumi

Wiki directory

Browse AI-generated documentation for any repository.

Architectural Overview

Relevant source files

The following files were used as context for generating this wiki page:

Pulumi's architecture is designed as a set of cooperating components that enable infrastructure as code in standard programming languages. This system allows developers to define and deploy cloud resources using languages like TypeScript, Python, Go, and C#, while Pulumi handles the state management, resource provisioning, and lifecycle operations.

The architecture is broadly composed of five main components: the deployment engine, state storage backends, language SDKs and hosts, resource providers, and a schema-driven code generation system. These components interact primarily through gRPC, with the Pulumi CLI acting as the main user entry point that orchestrates the entire process. In most deployments, the language host, CLI, and each resource provider run in separate processes.

Sources: developer-docs/architecture/overview.md

Core Components and Interaction

The Pulumi system is a distributed set of tools that communicate to execute a deployment. The central piece is the Pulumi CLI, which houses the deployment engine. The engine, in turn, communicates with two types of plugins: Language Hosts and Resource Providers.

The following diagram illustrates the high-level interaction between these core components.

Generating architecture diagram...

This diagram shows the flow of control, starting from the user invoking the Pulumi CLI, which triggers the engine to run the user's program via a language host, and finally uses a resource provider to interact with the target cloud API.

Sources: developer-docs/architecture/overview.md, sdk/nodejs/tests/runtime/langhost/run.spec.ts:468-522

The Deployment Engine

The deployment engine is the core of Pulumi, responsible for orchestrating the entire deployment lifecycle. It is embedded within the Pulumi CLI. Its primary responsibilities include:

Reading the project and stack configuration.
Starting the appropriate language host to run the user's infrastructure program.
Receiving resource registration requests from the user program.
Computing the difference between the desired state (from the program) and the current state (from the state file).
Loading and communicating with resource provider plugins to create, update, or delete resources.
Updating the state file in a backend.

The engine exposes a gRPC service (ResourceMonitor) that language runtimes use to register resources.

Sources: pkg/engine/engine.go, developer-docs/architecture/overview.md

Deployment Flow

A typical pulumi up command triggers a sequence of interactions between the engine, language host, and resource providers.

The following sequence diagram shows the gRPC calls during a resource registration.

Generating architecture diagram...

This sequence shows how a resource declared in user code results in gRPC calls from the user program to the engine, and from the engine to the appropriate provider plugin.

Sources: sdk/nodejs/tests/runtime/langhost/run.spec.ts:1030-1110

Language Ecosystem

Pulumi's multi-language support is enabled by a decoupled system of Language SDKs and Language Host plugins.

Language SDKs and Hosts

Language SDKs: These are the libraries (@pulumi/pulumi, pulumi, etc.) that users import in their code. They provide the core classes and functions like pulumi.Resource, pulumi.ComponentResource, and pulumi.Output. The SDK's primary job is to translate resource declarations into RegisterResource gRPC calls to the deployment engine.
Language Hosts: For each supported language, there is a corresponding language host plugin (e.g., pulumi-language-nodejs, pulumi-language-python). The engine starts the appropriate host for the project's defined runtime. The host is responsible for actually executing the user's program and providing a gRPC server that the engine communicates with.

The engine.ProjectInfoContext function in pkg/engine/engine.go is responsible for setting up the context needed to run a program, including loading the language runtime plugin.

Sources: pkg/engine/engine.go:275-296, sdk/nodejs/tests/runtime/langhost/run.spec.ts:1408-1422

Automation API

The Automation API is a programmatic interface that encapsulates the functionality of the deployment engine. It allows developers to embed Pulumi's deployment capabilities directly into their own applications, enabling use cases like CI/CD integration, custom CLIs, or multi-stack orchestration.

The API provides several ways to specify a Pulumi program:

NewStackLocalSource: For programs on the local filesystem.
NewStackRemoteSource: For programs in a Git repository.
NewStackInlineSource: For programs defined as a Go function within the automation code itself.

Sources: sdk/go/auto/README.md:19-37

Resource Providers

Resource providers are out-of-process gRPC plugins that are responsible for managing the lifecycle of resources for a specific cloud or service (e.g., AWS, Azure, Kubernetes). The deployment engine communicates with them to perform CRUD (Create, Read, Update, Delete) operations on resources.

The plugin.Host interface, implemented by the engine, is responsible for loading and managing these provider plugins. When the engine needs to perform an operation on a resource (e.g., aws:s3/bucket:Bucket), it ensures the corresponding provider plugin (pulumi-resource-aws) is running and forwards the request.

Sources: developer-docs/architecture/overview.md, pkg/resource/plugin/host.go

Schemas and Code Generation

A cornerstone of Pulumi's multi-language support is its schema-driven approach. Every resource provider is defined by a pulumi.json schema file. This schema is the source of truth for the provider's resources, their properties, and any provider-specific functions.

Package Schema (`pulumi.json`)

The package schema defines all aspects of a provider. The structure of this schema is defined by the metaschema.

Property	Description
`name`	The unqualified name of the package (e.g., "aws").
`version`	The semantic version of the package.
`description`	A Markdown description of the package.
`publisher`	The person or organization that published the package.
`license`	The license for the package's contents.
`pluginDownloadURL`	The URL to download the provider plugin binary from.
`provider`	A `resourceSpec` that defines the provider resource itself.
`config`	A map of configuration variables for the provider.
`resources`	A map from a type token to a `resourceSpec` for each resource.
`functions`	A map from a function token to a `functionSpec` for each function.
`types`	A map of complex (object) types used by resources and functions.
`language`	Language-specific overrides for code generation (e.g., C# namespaces, Python package names).

Sources: pkg/codegen/schema/pulumi.json:5-150

Code Generation

The language-specific SDKs for a provider are not written by hand; they are generated from the pulumi.json schema. A code generation tool reads the schema and outputs idiomatic code for each target language. This ensures consistency across languages and significantly reduces the effort to support new providers or resources.

The pkg/codegen/dotnet/gen.go file, for example, contains the logic to generate C# code from a schema. It uses templates to create .csproj files, resource classes, and input/output types.

The following diagram illustrates the code generation workflow.

Generating architecture diagram...

This process is what allows a single provider implementation to be used across all supported Pulumi languages.

Sources: pkg/codegen/dotnet/gen.go:1071-1088, developer-docs/architecture/overview.md

Project and State

Project Definition (`Pulumi.yaml`)

A Pulumi project is defined by a Pulumi.yaml file. This file specifies the project's name, the language runtime, and an optional description. The structure of this file is formally defined by a JSON schema.

Property	Description
`name`	The name of the project. Must be alphanumeric with hyphens, underscores, or periods.
`runtime`	The language runtime for the project (e.g., `nodejs`, `python`, `dotnet`, `go`). Can be a string or an object with options.
`description`	An optional description of the project.
`main`	An optional path to the Pulumi program's entry point. Defaults to the working directory.
`backend`	An optional object to explicitly set the backend URL.
`template`	If the project was created from a template, this section contains template metadata.

The pulumi new command orchestrates the creation of a new project, including the Pulumi.yaml file, based on a chosen template.

Sources: sdk/go/common/workspace/project.json:5-110, pkg/cmd/pulumi/newcmd/new.go:275-303

State Storage Backends

Pulumi uses a backend to store state files, which are JSON documents that represent the last-known state of your infrastructure. This allows Pulumi to compute diffs and manage resource lifecycles across multiple runs. Pulumi supports several backends:

The Pulumi Service (default)
Local filesystem (file://...)
Cloud object storage (e.g., s3://..., azblob://..., gs://...)

The backend is also responsible for secret encryption and management. The pulumi about command provides details on the currently configured backend.

Sources: developer-docs/architecture/overview.md, pkg/cmd/pulumi/about/about.go:204-219

Architectural Overview

Relevant source files

The following files were used as context for generating this wiki page:

Sources: developer-docs/architecture/overview.md

Core Components and Interaction

The following diagram illustrates the high-level interaction between these core components.

Generating architecture diagram...

Sources: developer-docs/architecture/overview.md, sdk/nodejs/tests/runtime/langhost/run.spec.ts:468-522

The Deployment Engine

The deployment engine is the core of Pulumi, responsible for orchestrating the entire deployment lifecycle. It is embedded within the Pulumi CLI. Its primary responsibilities include:

Reading the project and stack configuration.
Starting the appropriate language host to run the user's infrastructure program.
Receiving resource registration requests from the user program.
Computing the difference between the desired state (from the program) and the current state (from the state file).
Loading and communicating with resource provider plugins to create, update, or delete resources.
Updating the state file in a backend.

The engine exposes a gRPC service (ResourceMonitor) that language runtimes use to register resources.

Sources: pkg/engine/engine.go, developer-docs/architecture/overview.md

Deployment Flow

A typical pulumi up command triggers a sequence of interactions between the engine, language host, and resource providers.

The following sequence diagram shows the gRPC calls during a resource registration.

Generating architecture diagram...

This sequence shows how a resource declared in user code results in gRPC calls from the user program to the engine, and from the engine to the appropriate provider plugin.

Sources: sdk/nodejs/tests/runtime/langhost/run.spec.ts:1030-1110

Language Ecosystem

Pulumi's multi-language support is enabled by a decoupled system of Language SDKs and Language Host plugins.

Language SDKs and Hosts

Language SDKs: These are the libraries (@pulumi/pulumi, pulumi, etc.) that users import in their code. They provide the core classes and functions like pulumi.Resource, pulumi.ComponentResource, and pulumi.Output. The SDK's primary job is to translate resource declarations into RegisterResource gRPC calls to the deployment engine.
Language Hosts: For each supported language, there is a corresponding language host plugin (e.g., pulumi-language-nodejs, pulumi-language-python). The engine starts the appropriate host for the project's defined runtime. The host is responsible for actually executing the user's program and providing a gRPC server that the engine communicates with.

The engine.ProjectInfoContext function in pkg/engine/engine.go is responsible for setting up the context needed to run a program, including loading the language runtime plugin.

Sources: pkg/engine/engine.go:275-296, sdk/nodejs/tests/runtime/langhost/run.spec.ts:1408-1422

Automation API

The API provides several ways to specify a Pulumi program:

NewStackLocalSource: For programs on the local filesystem.
NewStackRemoteSource: For programs in a Git repository.
NewStackInlineSource: For programs defined as a Go function within the automation code itself.

Sources: sdk/go/auto/README.md:19-37

Resource Providers

Sources: developer-docs/architecture/overview.md, pkg/resource/plugin/host.go

Schemas and Code Generation

Package Schema (`pulumi.json`)

The package schema defines all aspects of a provider. The structure of this schema is defined by the metaschema.

Property	Description
`name`	The unqualified name of the package (e.g., "aws").
`version`	The semantic version of the package.
`description`	A Markdown description of the package.
`publisher`	The person or organization that published the package.
`license`	The license for the package's contents.
`pluginDownloadURL`	The URL to download the provider plugin binary from.
`provider`	A `resourceSpec` that defines the provider resource itself.
`config`	A map of configuration variables for the provider.
`resources`	A map from a type token to a `resourceSpec` for each resource.
`functions`	A map from a function token to a `functionSpec` for each function.
`types`	A map of complex (object) types used by resources and functions.
`language`	Language-specific overrides for code generation (e.g., C# namespaces, Python package names).

Sources: pkg/codegen/schema/pulumi.json:5-150

Code Generation

The pkg/codegen/dotnet/gen.go file, for example, contains the logic to generate C# code from a schema. It uses templates to create .csproj files, resource classes, and input/output types.

The following diagram illustrates the code generation workflow.

Generating architecture diagram...

This process is what allows a single provider implementation to be used across all supported Pulumi languages.

Sources: pkg/codegen/dotnet/gen.go:1071-1088, developer-docs/architecture/overview.md

Project and State

Project Definition (`Pulumi.yaml`)

Property	Description
`name`	The name of the project. Must be alphanumeric with hyphens, underscores, or periods.
`runtime`	The language runtime for the project (e.g., `nodejs`, `python`, `dotnet`, `go`). Can be a string or an object with options.
`description`	An optional description of the project.
`main`	An optional path to the Pulumi program's entry point. Defaults to the working directory.
`backend`	An optional object to explicitly set the backend URL.
`template`	If the project was created from a template, this section contains template metadata.

The pulumi new command orchestrates the creation of a new project, including the Pulumi.yaml file, based on a chosen template.

Sources: sdk/go/common/workspace/project.json:5-110, pkg/cmd/pulumi/newcmd/new.go:275-303

State Storage Backends

The Pulumi Service (default)
Local filesystem (file://...)
Cloud object storage (e.g., s3://..., azblob://..., gs://...)

The backend is also responsible for secret encryption and management. The pulumi about command provides details on the currently configured backend.

Sources: developer-docs/architecture/overview.md, pkg/cmd/pulumi/about/about.go:204-219

Architectural Overview

Core Components and Interaction

The Deployment Engine

Deployment Flow

Language Ecosystem

Language SDKs and Hosts

Automation API

Resource Providers

Schemas and Code Generation

Package Schema (pulumi.json)

Code Generation

Project and State

Project Definition (Pulumi.yaml)

State Storage Backends

Architectural Overview

Core Components and Interaction

The Deployment Engine

Deployment Flow

Language Ecosystem

Language SDKs and Hosts

Automation API

Resource Providers

Schemas and Code Generation

Package Schema (pulumi.json)

Code Generation

Project and State

Project Definition (Pulumi.yaml)

State Storage Backends

Package Schema (`pulumi.json`)

Project Definition (`Pulumi.yaml`)

Package Schema (`pulumi.json`)

Project Definition (`Pulumi.yaml`)