Data Tagging and Unsafe Proxies

Relevant source files

The following files were used as context for generating this wiki page:

• lib/ansible/utils/unsafe_proxy.py
• lib/ansible/_internal/_datatag/_tags.py
• test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py
• lib/ansible/modules/uri.py
• lib/ansible/module_utils/basic.py

The Data Tagging and Unsafe Proxy system is a core security feature in Ansible designed to handle sensitive data, such as passwords, keys, and tokens. Its primary purpose is to prevent the accidental disclosure of secrets in logs, console output, or other displays while allowing the data to be used transparently within playbook execution. This is achieved by wrapping sensitive values in special proxy objects that carry an "unsafe" tag.

This system is fundamental to Ansible's no_log feature. When a module parameter is marked with no_log: true, Ansible wraps the input value in an unsafe proxy. The system provides utilities to detect these tagged objects and sanitize them before they are displayed, typically by replacing the value with a placeholder like ANSIBLE_UNSAFE_VALUE.

Core Concepts

The system is built on two main components: data tagging classes and unsafe proxy objects.

1. Data Tagging: Simple marker classes are used to tag data as sensitive. The primary class for this is AnsibleUnsafe.
2. Unsafe Proxies: These are wrapper classes that behave like native Python types (e.g., str, bytes) but inherit from a data tagging class. This allows Ansible to treat the data as a normal string or bytes object for operational purposes, while also being able to identify it as sensitive.

The general workflow is as follows:

1. Sensitive data (e.g., from a no_log parameter or a vault-encrypted string) is wrapped in an AnsibleUnsafeText or AnsibleUnsafeBytes proxy object.
2. The proxy object is passed through the system, where it can be used by modules and plugins.
3. Before any data is logged or returned to the user, a sanitization function checks for objects tagged as AnsibleUnsafe.
4. If an object is tagged, its value is replaced with a placeholder string, preventing the secret from being exposed.

Sources: lib/ansible/utils/unsafe_proxy.py, lib/ansible/_internal/_datatag/_tags.py

Data Tagging Classes

The foundation of the system is a set of simple marker classes that are used for isinstance() checks to identify sensitive data.

Sources: lib/ansible/_internal/_datatag/_tags.py

Class	Description	File Path
AnsibleUnsafe	A generic marker class to indicate that a value is unsafe to display. This is the base for other tags.	lib/ansible/_internal/_datatag/_tags.py
AnsibleVaultEncryptedUnicode	A specific marker for vault-encrypted unicode strings. It inherits from AnsibleUnsafe.	lib/ansible/_internal/_datatag/_tags.py

A key utility function, is_unsafe(data), is provided to check if a given piece of data is an instance of AnsibleUnsafe.

# lib/ansible/_internal/_datatag/_tags.py:18-20
def is_unsafe(data):
    """check if a value is unsafe to display"""
    return isinstance(data, AnsibleUnsafe)

Sources: lib/ansible/_internal/_datatag/_tags.py:18-20

Unsafe Proxy Implementation

Unsafe proxies are the concrete implementation of data tagging. They are wrapper classes that mimic native Python types but are tagged as AnsibleUnsafe. This allows them to be used in most places where a string or bytes object is expected.

Sources: lib/ansible/utils/unsafe_proxy.py

Proxy Classes

Class	Inherits From	Description
AnsibleUnsafe	AnsibleUnsafe (tag)	A base proxy class that holds the original, unsafe data in self._unsafe_object.
AnsibleUnsafeBytes	bytes, AnsibleUnsafe	A proxy for byte strings. It behaves like a bytes object but is marked as unsafe.
AnsibleUnsafeText	str, AnsibleUnsafe	A proxy for text strings (unicode). It behaves like a str object but is marked as unsafe.

Class Structure

The following diagram illustrates the inheritance structure of the proxy classes.

Sources: lib/ansible/utils/unsafe_proxy.py, lib/ansible/_internal/_datatag/_tags.py

The proxies work by overriding methods like __repr__ to avoid displaying the sensitive content. For all other attribute access, they proxy the request to the wrapped native object (self._unsafe_object).

Sanitization

The primary use of data tagging is to enable sanitization of output. Functions that prepare data for display, such as module return values, use helpers to recursively scan data structures and replace any "unsafe" values.

Sanitization in Modules

The AnsibleModule class provides the exit_json and fail_json methods, which automatically sanitize return data. This is often done via a helper function like sanitize_keys. The uri module, for example, uses this to clean its results before exiting.

# lib/ansible/modules/uri.py:461-462
    if module.no_log_values:
        uresp = sanitize_keys(uresp, module.no_log_values, NO_MODIFY_KEYS)

Sources: lib/ansible/modules/uri.py:461-462

The sanitize_keys function, found in basic.py, recursively traverses dictionaries and lists, checking each value with is_unsafe and replacing it if necessary.

Sanitization Flow

This diagram shows the typical process of sanitizing module return data.

Sources: lib/ansible/module_utils/basic.py, lib/ansible/modules/uri.py

no_log and Argument Validation

The no_log parameter in a module's argument specification is the primary mechanism for a user to mark input as sensitive. When no_log: true is set for a parameter, Ansible automatically wraps its value in an AnsibleUnsafe proxy.

To enforce good security practice, Ansible includes a sanity test (validate-modules) that heuristically checks if a module argument should be marked with no_log.

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py

Heuristic Secret Detection

The is_potential_secret_option function contains the logic for identifying parameter names that are likely to be secrets.

# test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:133-151
def is_potential_secret_option(option_name):
    if not NO_LOG_REGEX.search(option_name):
        return False
    # If this is a count, type, algorithm, timeout, filename, or name, it is probably not a secret
    if option_name.endswith((
            '_count', '_type', '_alg', '_algorithm', '_timeout', '_name', '_comment',
            '_bits', '_id', '_identifier', '_period', '_file', '_filename',
    )):
        return False
    # 'key' also matches 'publickey', which is generally not secret
    if any(part in option_name for part in (
            'publickey', 'public_key', 'keyusage', 'key_usage', 'keyserver', 'key_server',
            'keysize', 'key_size', 'keyservice', 'key_service', 'pub_key', 'pubkey',
            'keyboard', 'secretary',
    )):
        return False
    return True

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:133-151

The logic can be summarized as:

1. The option name must match a regex for common secret-related terms (e.g., password, key, secret, token). This is defined by NO_LOG_REGEX.
2. The option name must NOT end with a suffix from a reject list that indicates a non-secret value (e.g., _file, _name, _id).
3. The option name must NOT contain substrings from a reject list that indicate a non-secret value (e.g., publickey, keyserver).

If an argument matches these criteria but does not have no_log: True in its specification, the validate-modules sanity test will raise a no-log-needed error.

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:2028-2037