Tokens and Lazy Resolution
Relevant source files
The following files were used as context for generating this wiki page:
Tokens are a core concept in the AWS CDK, acting as placeholders for values that are not available at the time of construct definition. Many values, such as the ARN of a resource you are defining, are only known after the CloudFormation template is synthesized or deployed. Tokens allow these values to be referenced and manipulated within the CDK application.
The process of replacing these tokens with their concrete, real-world values is called "resolution". This is a lazy process that occurs during the synthesis phase when the construct tree is walked and converted into a CloudFormation template. This system allows developers to work with values that will be resolved in the future, enabling the high-level, object-oriented abstractions that the CDK provides.
Core Concepts
IResolvable and Tokens
At the heart of the lazy resolution system is the IResolvable interface. Any object that implements this interface is considered a "token" or a "resolvable value". The interface defines a resolve method, which is called by the CDK framework during synthesis to obtain the concrete value.
IResolvable Interface:
export interface IResolvable {
/**
* The creation stack of this resolvable which will be appended to errors
* thrown during resolution.
*
* This may return an array of strings, a single string (as maintained by
* Error.stack), or undefined (in case stack frames are not available).
*/
readonly creationStack: string[];
/**
* Produce the Token's value at synthesis time.
*/
resolve(context: IResolveContext): any;
}
Sources: packages/@aws-cdk/core/lib/resolvable.ts:121-135
An Intrinsic is a common implementation of IResolvable that represents a value that can be resolved immediately, often wrapping a CloudFormation intrinsic function like { "Ref": "MyResource" }.
Sources: packages/@aws-cdk/core/test/tokens.test.ts:25-28
The Resolution Process
Resolution is performed by a resolver which traverses a given object structure (like a construct's properties) and resolves any IResolvable objects it finds. The main internal function for this is resolve() located in packages/@aws-cdk/core/lib/private/resolve.ts.
The process can be visualized as follows:
This diagram shows the recursive nature of the resolve function as it walks through different data types.
Sources: packages/@aws-cdk/core/lib/private/resolve.ts:63-220
The IResolveContext passed to a token's resolve method provides information about the current scope and allows for further resolution.
IResolveContext Property | Description |
|---|---|
scope | The construct in which the resolution is occurring. |
preparing | A boolean indicating if the resolution is for pre-synthesis preparation. |
resolve() | A function to recursively resolve a value within the current context. |
registerPostProcessor() | Allows a token to register a post-processor to modify the resolved value. |
Sources: packages/@aws-cdk/core/lib/resolvable.ts:141-163
Token Types and Encoding
To allow tokens to be used in primitive types like strings and numbers, the CDK uses an encoding mechanism.
String Tokens
When a token is used within a string literal (e.g., `arn:aws:s3:::${bucket.bucketName}`), it is stringified into a special format like ${Token[TOKEN.123]}. During resolution, these markers are found, and the string is split into literals and token parts. The resolver then typically uses the Fn::Join CloudFormation intrinsic to combine the parts.
// Example from tests
const token = new Intrinsic( ({ woof: 'woof' }));
const stringified = `The dog says: ${token}`;
const resolved = resolve(stringified);
// `resolved` will be:
// {
// 'Fn::Join': ['', ['The dog says: ', { woof: 'woof' }]],
// }
Sources: packages/@aws-cdk/core/test/tokens.test.ts:135-148
The Tokenization.reverseString() function can be used to extract a token from its stringified representation.
Sources: packages/@aws-cdk/core/test/tokens.test.ts:125-133
List Tokens
A token that resolves to a list of strings can be represented using Token.asList(value). This returns an encoded string array.
Important constraints:
- You cannot add new elements to the returned array (e.g., via
push()). - You cannot modify string elements within the array.
- Accessing an element by index (e.g.,
list[0]) will fail at resolution time. UseFn.select(0, list)instead.
Sources: packages/@aws-cdk/core/test/tokens.test.ts:311-351
Number Tokens
A token that resolves to a number is encoded using Token.asNumber(value). This returns a special numeric value that the resolver can identify and decode. The internal mechanism uses createTokenDouble and extractTokenDouble to manage this encoding. This allows numbers that are actually unresolved tokens to be passed around where a number type is expected.
Sources: packages/@aws-cdk/core/lib/token.ts:119-126, packages/@aws-cdk/core/test/tokens.test.ts:389-423
Lazy Evaluation
The Lazy class provides a way to create tokens whose values are produced by a function that is only executed when the token is resolved. This is useful for deferring complex calculations or lookups until synthesis time.
Class Diagram
This diagram shows that the static methods on the Lazy class produce values (either IResolvable objects or encoded primitives) that participate in the resolution system.
Sources: packages/@aws-cdk/core/lib/lazy.ts, packages/@aws-cdk/core/lib/resolvable.ts:121-135
Cached vs. Uncached Producers
The Lazy class offers both cached and uncached value producers.
| Method | Caching Behavior | Description |
|---|---|---|
Lazy.any() | Cached | The produce function is called only once. The result is cached and returned for all subsequent resolutions. |
Lazy.string() | Cached | Same as Lazy.any(), but returns an encoded string token. |
Lazy.number() | Cached | Same as Lazy.any(), but returns an encoded number token. |
Lazy.list() | Cached | Same as Lazy.any(), but returns an encoded list token. |
Lazy.uncachedString() | Uncached | The produce function is executed every time the token is resolved. |
Sources: packages/@aws-cdk/core/lib/lazy.ts:25-103, packages/@aws-cdk/core/test/tokens.test.ts:245-269
The following sequence diagram illustrates how a cached lazy value is resolved.
This diagram shows that the producer function is only called on the first resolution.
Sources: packages/@aws-cdk/core/test/tokens.test.ts:257-269
Debugging
To aid in debugging resolution errors, every token captures its creation stack trace. If an error occurs during resolution, the framework can append this stack trace to the error message, pointing developers to where the problematic token was defined in their code.
The newError() method on a token can be used to throw an error that automatically includes this contextual information.
This behavior is controlled by the CDK_DEBUG environment variable. When CDK_DEBUG is set to 'true', the creation stack trace is included in resolution error messages.
Sources: packages/@aws-cdk/core/lib/resolvable.ts:127-132, packages/@aws-cdk/core/test/tokens.test.ts:425-452, 513-549
Summary
Tokens and Lazy Resolution are fundamental mechanisms in the AWS CDK. They create a bridge between the imperative programming model used to define infrastructure and the declarative, resolved values needed for CloudFormation synthesis. By using tokens, encoded primitives, and lazy producers, the CDK allows developers to compose high-level abstractions with values that will only be known in the future, making the entire framework powerful and flexible.