Assets Handling (Files & Docker Images)

Relevant source files

The following files were used as context for generating this wiki page: - [packages/@aws-cdk/aws-s3-assets/README.md](https://github.com/fwang/aws-cdk/blob/HEAD/packages/%40aws-cdk/aws-s3-assets/README.md) - [packages/@aws-cdk/aws-s3-assets/test/asset.test.ts](https://github.com/fwang/aws-cdk/blob/HEAD/packages/%40aws-cdk/aws-s3-assets/test/asset.test.ts) - [packages/@aws-cdk/cloud-assembly-schema/schema/assets.schema.json](https://github.com/fwang/aws-cdk/blob/HEAD/packages/%40aws-cdk/cloud-assembly-schema/schema/assets.schema.json) - [packages/cdk-assets/README.md](https://github.com/fwang/aws-cdk/blob/HEAD/packages/cdk-assets/README.md) - [packages/aws-cdk/lib/assets.ts](https://github.com/fwang/aws-cdk/blob/HEAD/packages/aws-cdk/lib/assets.ts) - [packages/@aws-cdk/pipelines/lib/codepipeline/codepipeline.ts](https://github.com/fwang/aws-cdk/blob/HEAD/packages/%40aws-cdk/pipelines/lib/codepipeline/codepipeline.ts) - [packages/@aws-cdk/pipelines/test/compliance/assets.test.ts](https://github.com/fwang/aws-cdk/blob/HEAD/packages/%40aws-cdk/pipelines/test/compliance/assets.test.ts) - [packages/@aws-cdk/aws-ec2/test/cfn-init.test.ts](https://github.com/fwang/aws-cdk/blob/HEAD/packages/%40aws-cdk/aws-ec2/test/cfn-init.test.ts) - [packages/@aws-cdk/core/test/stack-synthesis/new-style-synthesis.test.ts](https://github.com/fwang/aws-cdk/blob/HEAD/packages/%40aws-cdk/core/test/stack-synthesis/new-style-synthesis.test.ts)

Assets are local files, directories, or Docker images that are bundled and deployed alongside a CDK application. They are essential for packaging application code, configuration files, or any other artifacts required by the infrastructure at runtime. The CDK framework provides a streamlined mechanism for handling these assets, from local packaging to deployment in AWS.

The asset handling system is primarily divided into two categories: file-based assets, which are uploaded to Amazon S3, and Docker image assets, which are pushed to Amazon ECR. The entire process is managed through an asset manifest file (assets.json) generated during synthesis, and a publishing tool (cdk-assets) that processes this manifest.

Sources: packages/@aws-cdk/aws-s3-assets/README.md:5-8, packages/cdk-assets/README.md:5-13

Asset Lifecycle and Core Concepts

The lifecycle of an asset involves three main phases: definition in the CDK app, synthesis into a cloud assembly, and publishing during deployment.

This diagram illustrates the end-to-end process of handling assets in the AWS CDK.
Sources: packages/@aws-cdk/aws-s3-assets/README.md:30-41, packages/cdk-assets/README.md:15-20

Asset Manifest (assets.json)

During the cdk synth command, the CDK generates a cloud assembly in the cdk.out directory. A key component of this assembly is the assets.json file. This manifest lists all file and Docker image assets required by the application, detailing their source on the local disk and their intended destinations in AWS. The cdk-assets tool uses this manifest to perform the publishing operations.

The schema for the manifest defines two main types of assets: files and dockerImages. Each asset entry contains a source block describing how to package the asset and a destinations block describing where to publish it.

Sources: packages/cdk-assets/README.md:15-20, packages/@aws-cdk/cloud-assembly-schema/schema/assets.schema.json:4-22

The following diagram shows the high-level structure of the asset manifest.

Asset Staging

When cdk synth is executed, the CDK framework stages the assets by copying them from their original location into the cdk.out directory.

• File assets are copied to a path like cdk.out/asset.<hash>.<ext>.
• Directory assets are copied into a directory like cdk.out/asset.<hash>/.

This ensures that the cloud assembly is self-contained and all necessary files for deployment are located in one place. The path in the assets.json manifest points to this staged location. If staging is disabled (via context key DISABLE_ASSET_STAGING_CONTEXT), the manifest will point to the original absolute path.

Sources: packages/@aws-cdk/aws-s3-assets/test/asset.test.ts:282-358

Asset Publishing (cdk-assets tool)

The cdk-assets command-line tool is responsible for publishing the assets defined in assets.json to their respective AWS destinations. It reads the manifest, prepares the assets (e.g., zipping directories), and uploads them to S3 or ECR. It is designed to be idempotent; if an asset with the same hash already exists at the destination, it will not be re-uploaded.

The tool can be invoked to publish all assets or a subset by specifying asset or destination IDs.

# Example usage
cdk-assets /path/to/cdk.out [ASSET:DEST] [ASSET] [:DEST] [...]

Sources: packages/cdk-assets/README.md:22-30, packages/cdk-assets/package.json:6-9

File & Directory Assets (@aws-cdk/aws-s3-assets)

File and directory assets are managed by the @aws-cdk/aws-s3-assets module. The primary construct is Asset, which represents a local file or directory to be uploaded to an S3 bucket.

Defining an Asset

An Asset is defined by providing a path to the local file or directory. The CDK determines whether it's a file or directory and sets the packaging accordingly.

• Directories are packaged as .zip files.
• Single files are uploaded as-is.

import { Asset } from '@aws-cdk/aws-s3-assets';
import * as path from 'path';

// Asset from a directory, will be zipped
const directoryAsset = new Asset(this, 'ZipAsset', {
  path: path.join(__dirname, 'sample-asset-directory'),
});

// Asset from a single file
const fileAsset = new Asset(this, 'FileAsset', {
  path: path.join(__dirname, 'file-asset.txt'),
});

Sources: packages/@aws-cdk/aws-s3-assets/README.md:10-18, packages/@aws-cdk/aws-s3-assets/test/asset.test.ts:153-162

Deploy-Time Attributes

The Asset construct exposes several attributes that can be used to reference the asset's location in S3 after deployment. These are resolved at deploy time via CloudFormation parameters.

Attribute	Description	Example
s3BucketName	The name of the S3 bucket where the asset is stored.	my-assets-bucket
s3ObjectKey	The S3 object key of the asset file.	123abcde.../asset.zip
s3ObjectUrl	The S3 URL of the asset.	s3://my-assets-bucket/123abcde.../asset.zip
httpUrl	The S3 HTTP URL of the asset.	https://s3.region.amazonaws.com/.../asset.zip

Sources: packages/@aws-cdk/aws-s3-assets/README.md:22-27

Permissions

To grant an IAM principal (like a role or user) read access to an asset in S3, use the grantRead() method. This will add the necessary s3:GetObject*, s3:GetBucket*, and s3:List* permissions to the principal's policy.

const asset = new Asset(stack, 'MyAsset', { path: '...' });
const user = new iam.User(stack, 'MyUser');
asset.grantRead(user);

Sources: packages/@aws-cdk/aws-s3-assets/README.md:33-37, packages/@aws-cdk/aws-s3-assets/test/asset.test.ts:110-132

Asset Bundling

The bundling option allows for custom processing of assets within a Docker container before packaging. The source asset directory is mounted at /asset-input, and the bundling command is expected to write its output to /asset-output. The contents of /asset-output are then zipped and uploaded to S3.

This is useful for compiling code, converting file formats, or any other build step.

new assets.Asset(this, 'BundledAsset', {
  path: '/path/to/asset',
  bundling: {
    image: DockerImage.fromRegistry('alpine'),
    command: ['sh', '-c', 'echo "hello" > /asset-output/output.txt'],
  },
});

A local bundling provider can also be specified via the local property to improve performance by avoiding Docker when possible.

Sources: packages/@aws-cdk/aws-s3-assets/README.md:58-70, packages/@aws-cdk/aws-s3-assets/README.md:83-96

Docker Image Assets (@aws-cdk/aws-ecr-assets)

Docker image assets are managed by the @aws-cdk/aws-ecr-assets module. The DockerImageAsset construct builds a Docker image from a local directory containing a Dockerfile and pushes it to a repository in Amazon ECR.

Defining a Docker Image Asset

To define a Docker image asset, you provide the path to the directory containing the Dockerfile and build context.

import { DockerImageAsset } from '@aws-cdk/aws-ecr-assets';

const imageAsset = new DockerImageAsset(this, 'MyBuildImage', {
  directory: path.join(__dirname, 'my-docker-image'),
});

Sources: packages/@aws-cdk/aws-ecr-assets/package.json:4-5, packages/aws-cdk/lib/assets.ts:101-105

During synthesis, the CDK will:

1. Calculate a hash of the source directory.
2. Add an entry to the assets.json manifest specifying the source and build options.
3. During deployment, the cdk-assets tool will build the Docker image and tag it with the source hash.
4. The tool then pushes the image to a dedicated ECR repository.

The imageUri attribute of the asset will resolve to the full ECR URI of the pushed image, which can be used in constructs like ECS Task Definitions.

Sources: packages/aws-cdk/lib/assets.ts:101-125, packages/@aws-cdk/core/test/stack-synthesis/new-style-synthesis.test.ts:121-131

Integration with CloudFormation

Assets are seamlessly integrated with CloudFormation through a combination of construct metadata and template parameters.

1. Metadata: When an Asset is defined, a metadata entry of type aws:cdk:asset is added to the construct tree. This entry contains the asset's path, packaging type, and hash.
2. Parameters: The synthesized CloudFormation template includes parameters for the asset's S3 bucket (s3BucketParameter) and object key (s3KeyParameter).
3. Deployment: During cdk deploy, the CDK toolkit first publishes the asset to S3/ECR. It then passes the actual bucket name and object key/image URI as values for the CloudFormation parameters when deploying the stack.

The following sequence diagram shows the interaction during a cdk deploy command.

Assets in CDK Pipelines

When using CDK Pipelines (@aws-cdk/pipelines), asset handling is automated. The pipeline automatically adds an "Assets" stage before any deployment stages.

• This stage contains one or more CodeBuild projects responsible for publishing all file and Docker image assets.
• By default, assets are published in parallel to improve speed (publishAssetsInParallel: true). This can be changed to a single publisher per asset type.
• The pipeline creates dedicated IAM roles (CdkAssetsFileRole, CdkAssetsDockerRole) for the CodeBuild projects, which are granted permissions to assume the publishing roles in the target environments.

This ensures that all assets are available in the target accounts and regions before the application stacks are deployed.

Sources: packages/@aws-cdk/pipelines/lib/codepipeline/codepipeline.ts:311-320, packages/@aws-cdk/pipelines/test/compliance/assets.test.ts:80-92, packages/@aws-cdk/pipelines/lib/codepipeline/codepipeline.ts:503-529

Conclusion

The AWS CDK asset system provides a robust and automated way to manage local files and Docker images as part of your infrastructure-as-code. By abstracting away the details of packaging, hashing, and uploading, it allows developers to focus on application logic while ensuring that deployments are repeatable and reliable. The integration with CloudFormation and CDK Pipelines makes it a powerful feature for building and deploying complex applications.