Dynamic Field Handling

Relevant source files

The following files were used as context for generating this wiki page:

• internal/runner/dynamic_fields.go
• internal/config/config.go
• internal/runner/matcher.go
• internal/tui/onboard/config.go
• internal/tui/onboard/view.go
• internal/tui/onboard-cloud/save.go

Dynamic Field Handling is a core feature of the Tusk Drift test runner that allows for deterministic comparison of JSON responses by ignoring fields that are expected to change between test runs. This is essential for values like timestamps, UUIDs, and other dynamically generated data. By defining rules in the configuration, the test runner can validate the structure and static content of a response while gracefully skipping over these volatile fields.

This system is implemented through the DynamicFieldMatcher, which is initialized with rules from the .tusk/config.yaml file. During the replay phase, when the runner compares a recorded response with the actual response from the service under test, it uses this matcher to decide whether a mismatch in a specific field should be ignored or reported as a test failure.

Configuration

The behavior of the dynamic field matcher is controlled via the comparison.dynamic_fields section in the .tusk/config.yaml file. This section allows you to specify JSON paths and regular expressions to identify fields that should be ignored during comparison.

Sources: internal/config/config.go:102-105

YAML Structure

The configuration is defined by the DynamicFields struct.

// internal/config/config.go:102-105
type DynamicFields struct {
	Paths []string `mapstructure:"paths"`
	Regex []string `mapstructure:"regex"`
}

Sources: internal/config/config.go:102-105

This corresponds to the following structure in config.yaml:

comparison:
  dynamic_fields:
    paths:
      - "data.id"
      - "data.items[*].createdAt"
    regex:
      - ".*Timestamp"
      - ".*_at"

Configuration Options

The following table describes the available options under comparison.dynamic_fields.

Key	Type	Description
paths	[]string	A list of JSON path strings. Supports exact paths and glob patterns (e.g., data.items[*].id) to match specific fields.
regex	[]string	A list of regular expression strings. Any JSON path that matches one of these regexes will be considered dynamic.

Sources: internal/config/config.go:103-104

Matching Architecture

The core of the dynamic field handling is the DynamicFieldMatcher. This component is responsible for parsing the configuration and providing a simple interface to check if a given JSON path is dynamic.

Sources: internal/runner/dynamic_fields.go

DynamicFieldMatcher Struct

The DynamicFieldMatcher holds the compiled rules from the configuration.

// internal/runner/dynamic_fields.go:15-20
type DynamicFieldMatcher struct {
	exactPaths map[string]struct{}
	globPaths  []glob.Glob
	regexPaths []*regexp.Regexp
}

• exactPaths: A set for O(1) lookups of exact JSON paths.
• globPaths: A slice of compiled glob patterns for wildcard matching.
• regexPaths: A slice of compiled regular expressions for flexible path matching.

Sources: internal/runner/dynamic_fields.go:15-20

Initialization Flow

The NewDynamicFieldMatcher function initializes the matcher by taking the DynamicFields configuration, compiling the glob and regex patterns, and separating exact paths for efficient lookup.

This diagram illustrates the initialization process.

Matching Logic (IsDynamic method)

The primary method is IsDynamic(path string), which checks if a given JSON path string should be considered dynamic. The logic follows a specific order of precedence:

1. Exact Match: It first checks for an exact match in the exactPaths map.
2. Glob Match: If no exact match is found, it iterates through the compiled globPaths to see if any pattern matches the path.
3. Regex Match: Finally, if no glob matches, it tests the path against the compiled regexPaths.

If any of these checks return true, the path is considered dynamic.

The following diagram shows the decision flow within the IsDynamic method.

Integration in Test Runner

The DynamicFieldMatcher is used within the test runner's bodyMatches function to compare the JSON bodies of the recorded and actual HTTP responses.

When deep.Equal reports a difference, the runner extracts the JSON path of the differing field and passes it to dynamicMatcher.IsDynamic(). If the method returns true, the difference is ignored, and the test continues. If it returns false, the difference is recorded as a test failure.

This diagram shows how the matcher is consulted upon finding a discrepancy.

Sources: internal/runner/matcher.go:108-115

Configuration Management

While the comparison.dynamic_fields configuration is powerful, it is intended to be set up manually by the user in .tusk/config.yaml.

Initial Generation

The tusk init command and its associated onboarding wizard generate the initial .tusk/config.yaml file. However, the wizard does not prompt for dynamic field paths. It creates a default configuration structure, and users are expected to add the dynamic_fields section themselves.

The final YAML file is formatted using utilities that ensure proper indentation and spacing, such as formatYAMLWithBlankLines.

Sources: internal/tui/onboard/config.go:34-60, internal/tui/onboard/view.go:90-95

Programmatic Updates

The codebase contains logic for programmatically updating the YAML configuration file while preserving comments and structure. The updateYAMLNode function in internal/tui/onboard-cloud/save.go demonstrates how to traverse the YAML abstract syntax tree (AST) and modify or add nodes.

This capability is used in the cloud onboarding flow but could be extended to manage dynamic fields in the future. It works by unmarshalling the YAML into a yaml.Node, finding or creating the target path, and then updating its value.

// internal/tui/onboard-cloud/save.go:101-104
// updateYAMLNode finds or creates the nested path in the YAML node and sets the final key to the given value.
func updateYAMLNode(root *yaml.Node, updates []configUpdate) error {
    // ... logic to traverse and update yaml.Node
}

Sources: internal/tui/onboard-cloud/save.go:101-149

Conclusion

Dynamic Field Handling is a critical mechanism in tusk-drift-cli that ensures the stability and reliability of replay tests. By providing flexible configuration options through paths (with glob support) and regex, developers can precisely define which parts of a response are expected to change, allowing the test runner to focus on verifying the parts that should remain constant. The DynamicFieldMatcher provides an efficient, multi-layered approach to applying these rules during test execution.