Building From Source

Building the Pulumi project from source involves several distinct processes, including compiling the core Go-based CLI, generating language-specific SDKs from a shared schema, and building the developer documentation. The entire build and contribution workflow is orchestrated using a Makefile and various scripts, providing a consistent experience for developers.

This document provides a high-level overview of the build system. For more detailed instructions on setting up a development environment, refer to the Pulumi Developer Documentation.

Sources: CONTRIBUTING.md, README.md, developer-docs/index.md

Contribution Workflow

The standard contribution process involves setting up a development environment, making code changes, running validation checks, and submitting a pull request.

Sources: CONTRIBUTING.md:37-41

Development and Validation

Before submitting a pull request, all lint checks must pass. The project uses gofumpt for Go code formatting.

Sources: CONTRIBUTING.md:44-57

The following diagram illustrates the typical developer workflow for contributing to the Pulumi repository.

Sources: CONTRIBUTING.md:37-60

Language SDK Generation

Pulumi generates SDKs for multiple languages from a common Pulumi Schema. This schema-driven approach ensures consistency across different language implementations. The generation logic for each language is contained within the pkg/codegen/ directory.

Sources: pkg/codegen/dotnet/gen.go, pkg/codegen/python/gen.go

.NET SDK Generation

The .NET SDK is generated by the GeneratePackage function, which orchestrates the creation of all necessary C# project files.

Key components of the .NET code generation process include:

• Module Context (modContext): A struct that holds the context for a specific module, including its types, resources, functions, and namespace information.
• GeneratePackage: The main entry point that drives the generation of the entire .NET package, including project files, source code, and metadata.
• genProjectFile: Generates the .csproj file, defining package metadata, dependencies (PackageReference), and local project references (ProjectReference).
• genUtilities: Creates a Utilities.cs file containing helper methods, version information, and plugin download URLs, based on a Go template.
• genResource: Generates a C# class for each Pulumi resource, including its input arguments (...Args), output properties, and methods.

Sources: pkg/codegen/dotnet/gen.go:2101-2122, pkg/codegen/dotnet/gen.go:1730-1763, pkg/codegen/dotnet/gen.go:1680-1710, pkg/codegen/dotnet/gen.go:1078-1383

The following diagram shows the flow for generating a .NET package from a Pulumi schema.

Sources: pkg/codegen/dotnet/gen.go:2101-2122, pkg/codegen/dotnet/templates.go

Python SDK Generation

The Python SDK generation follows a similar pattern, managed by the GeneratePackage function in the Python codegen directory.

Key components include:

• GeneratePackage: The main function that orchestrates the creation of a Python package.
• genPackageMetadata / genPyprojectTOML: Generates setup.py or pyproject.toml respectively, defining package metadata and dependencies found in install_requires.
• genUtilitiesFile: Creates the _utilities.py file with helper functions.
• genResource: Generates a Python class for each resource, its ...Args class, and methods.
• __init__.py Generation: Each module and submodule gets an __init__.py file, which handles lazy-loading of submodules and registration of resource mappings.

Sources: pkg/codegen/python/gen.go:2759-2788, pkg/codegen/python/gen.go:2150-2244, pkg/codegen/python/gen.go:722-758, pkg/codegen/python/gen.go:1329-1602, pkg/codegen/python/gen.go:949-1033

The following diagram shows the flow for generating a Python package from a Pulumi schema.

Sources: pkg/codegen/python/gen.go:2759-2788, tests/testdata/codegen/simple-resource-schema/python/setup.py

Documentation Build

The project includes developer-focused documentation, which has its own build process separate from the core CLI and SDKs.

Sources: developer-docs/build.md

Documentation Tools and Prerequisites

The developer documentation is built using a specific set of tools.

Sources: developer-docs/build.md:3-14

Building the Documentation

The documentation can be built using make from either the developer-docs directory or the repository root.

Sources: developer-docs/build.md:16-30

The documentation build process is as follows:

Sources: developer-docs/build.md:10-21

Releasing and Versioning

The release process is a semi-automated workflow that involves version bumping, running tests, and publishing a release on GitHub.

The key steps are:

1. Freeze the Release: A developer runs the scripts/set-version.py <next-patch-version> script. This updates sdk/.version and stops the current draft release from being updated further.
2. Run Tests: To ensure quality, the pulumi/examples and pulumi/templates test suites are run against the latest development version.
3. Publish Release: A maintainer navigates to the GitHub Releases page, edits the frozen draft, unticks "Set as a pre-release", and publishes it.
4. Auto-Merge Cleanup: After publishing, pulumi-bot creates a pull request with go.mod updates and changelog cleanups, which is then auto-merged.

Sources: CONTRIBUTING.md:112-148

The following diagram illustrates the release sequence.

Sources: CONTRIBUTING.md:112-148