Build and Test Process

The AWS Cloud Development Kit (CDK) utilizes a sophisticated, multi-layered build and test process tailored for a large TypeScript monorepo. The system relies on a combination of npm/yarn scripts, a centralized package of build tools (cdk-build-tools), and specialized scripts for code generation and packaging. This process ensures consistency across the many modules of the CDK, from core libraries to individual service constructs.

At a high level, the process involves compiling TypeScript source code, running unit and integration tests, linting, and packaging modules for distribution across multiple language ecosystems (JavaScript/TypeScript, Python, Java, .NET, Go) using jsii. The cdk CLI itself provides commands like synth, deploy, and diff which are central to the developer workflow but are distinct from the repository's internal build process.

Core Build Tooling: cdk-build-tools

The foundation of the build system is the cdk-build-tools package. This internal package provides a set of executable scripts that standardize common development tasks across all packages in the monorepo.

Sources: tools/cdk-build-tools/package.json

Provided Binaries

The bin section of cdk-build-tools/package.json defines the following command-line tools:

Sources: tools/cdk-build-tools/package.json:11-19

These tools are invoked by the scripts section in the package.json of each individual module, providing a consistent interface for developers.

Package-Level Build Scripts

Each package within the AWS CDK repository contains a package.json file with a scripts section that orchestrates the build and test lifecycle for that specific module. These scripts delegate to the binaries provided by cdk-build-tools.

Sources: packages/aws-cdk/package.json, packages/@aws-cdk/aws-s3-assets/package.json, packages/@aws-cdk/pipelines/package.json

Common Scripts

The following diagram illustrates the typical flow when a developer runs a build or test command within a package.

This diagram shows how yarn scripts in a package's package.json map to the centralized cdk-build-tools commands.
Sources: tools/cdk-build-tools/package.json:11-19, packages/aws-cdk/package.json:9-25

A typical scripts section looks like this:

"scripts": {
  "build": "cdk-build",
  "watch": "cdk-watch",
  "lint": "cdk-lint",
  "test": "cdk-test",
  "integ": "cdk-integ",
  "pkglint": "pkglint -f",
  "package": "cdk-package",
  "awslint": "cdk-awslint",
  "build+test+package": "yarn build+test && yarn package",
  "build+test": "yarn build && yarn test",
  "compat": "cdk-compat",
  "rosetta:extract": "yarn --silent jsii-rosetta extract",
  "build+extract": "yarn build && yarn rosetta:extract",
  "build+test+extract": "yarn build+test && yarn rosetta:extract"
}

Sources: packages/@aws-cdk/aws-s3-assets/package.json:24-38

This structure allows for complex, chained commands like build+test+package that ensure code is built and tested before being packaged.

Build Specifications for CI/CD

The AWS CDK project uses AWS CodeBuild, and its build process is defined using BuildSpec objects. The repository includes tests for how these specifications are created and merged, demonstrating a flexible approach to defining CI/CD pipelines.

Sources: packages/@aws-cdk/aws-codebuild/test/test.build-spec.ts

BuildSpec Merging

The mergeBuildSpecs function allows combining multiple BuildSpec configurations. This is useful for layering a generic build process with a user-defined one.

The merging logic is as follows:

• Phases: Commands within the same phase (e.g., build, pre_build) are concatenated.
• Environment Variables: variables and parameter-store entries are merged.
• Reports: Report configurations are merged. If a report has the same key in both specs, the right-hand side (RHS) spec's configuration overwrites the left-hand side (LHS).
• Artifacts: The artifacts section is not merged; if both specs define it, an error is thrown to prevent ambiguity.

Sources: packages/@aws-cdk/aws-codebuild/test/test.build-spec.ts:15-21, 62-72, 85-88, 222-258

The following diagram shows the merging of two BuildSpec objects.

This diagram illustrates how commands from the pre_build phase and environment variables from two separate BuildSpec objects are combined into a single, merged specification.
Sources: packages/@aws-cdk/aws-codebuild/test/test.build-spec.ts:104-219

Specialized Build and Generation Scripts

Beyond standard compilation and testing, the repository contains specialized scripts for code generation tasks that are critical to the CDK's functionality.

CloudFormation Spec Generation

The packages/@aws-cdk/cfnspec module has a build process to generate a unified CloudFormation resource specification.

The process is as follows:

1. Read all .json files from the spec-source directory.
2. Files are sorted and processed.
3. Specification fragments are merged into a single spec object.
4. Patch files (*patch.json) are applied to the merged spec.
5. The final specification is massaged and normalized.
6. An MD5 fingerprint of the normalized spec is calculated and stored.
7. The final specification.json is written to the spec directory.

This flowchart shows the steps involved in creating the master CloudFormation specification file used by the CDK.
Sources: packages/@aws-cdk/cfnspec/build-tools/build.ts:19-40

Automatic Module Creation

The create-missing-libraries.ts script automates the creation of new CDK construct library modules. It iterates through all CloudFormation namespaces and checks if a corresponding CDK package exists. If not, it generates a complete package structure, including:

• package.json with correct dependencies and build scripts.
• .gitignore and .npmignore.
• lib/index.ts exporting the generated resources.
• A placeholder test file.
• A README.md file.
• jest and eslint configuration files.

This automation is crucial for keeping the AWS CDK in sync with new services and resources added to AWS CloudFormation.

Sources: packages/@aws-cdk/cfnspec/build-tools/create-missing-libraries.ts:16-189

User Project Initialization (cdk init)

The cdk init command is part of the aws-cdk package and prepares a new user project for its own build and test lifecycle. It uses templates to scaffold a new CDK application in a chosen language.

The initialization process:

1. Selects a template (e.g., app, sample-app) and language.
2. Copies files from the template directory.
3. Processes template files (*.template.*), replacing placeholders like %name% and %cdk-version%.
4. Adds current FUTURE_FLAGS to the cdk.json context to enable new features by default.
5. Invokes post-installation hooks (*.hook.js) for language-specific setup.
6. Initializes a new Git repository.
7. Runs package manager installations (e.g., npm install) for Node.js projects.

This sequence diagram shows the steps executed by cdk init to create a new, ready-to-use CDK project.
Sources: packages/aws-cdk/lib/init.ts:28-44, 260-269, 275-283, 285-305