Developing Lookup, Filter, and Test Plugins

Relevant source files

The following files were used as context for generating this wiki page:

• lib/ansible/plugins/lookup/file.py
• lib/ansible/plugins/filter/core.py
• lib/ansible/plugins/test/core.py
• lib/ansible/cli/doc.py
• test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py
• test/integration/targets/ansible-doc/test.yml
• test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt
• lib/ansible/modules/lineinfile.py

Lookup, Filter, and Test plugins are essential for extending Ansible's data manipulation and validation capabilities within playbooks. Lookups are used to fetch data from external sources, while Filters and Tests are used to manipulate and validate data within Jinja2 templates, respectively. Developing these plugins requires adherence to specific structural conventions, documentation standards, and implementation patterns to ensure they are correctly loaded, documented, and validated by the Ansible engine.

New plugins are generally expected to be developed within Ansible Collections, not ansible-core. This modular approach allows for better organization and distribution of custom functionality. All plugin code must be GPLv3 compatible.

Sources: AGENTS.md:12, 219, README.md:10, 102, test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt

Plugin Architecture and Structure

Plugins are organized by type within a collection's plugins directory. This structure is crucial for Ansible's plugin loader to discover and register them correctly.

Sources: test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt:12-14, 28, 38, 68

Directory Structure

Within an Ansible Collection, these plugins reside in specific subdirectories:

Plugin Type	Directory
Lookup	plugins/lookup/
Filter	plugins/filter/
Test	plugins/test/

Sources: test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt:12, 38, 68

Core Implementation

Each plugin type has a distinct base class and primary method that must be implemented.

Plugin Type	Base Class	Primary Method	Purpose
Lookup	LookupBase	run(terms, variables=None, **kwargs)	Executes the lookup and returns a list of values.
Filter	FilterModule	filters()	Returns a dictionary of filter names mapped to their corresponding functions.
Test	TestModule	tests()	Returns a dictionary of test names mapped to their corresponding functions.

Sources: lib/ansible/plugins/lookup/file.py, lib/ansible/plugins/filter/core.py, lib/ansible/plugins/test/core.py

The following diagram illustrates the basic class structure for each plugin type.

Sources: lib/ansible/plugins/lookup/file.py, lib/ansible/plugins/filter/core.py, lib/ansible/plugins/test/core.py

Documentation

Proper documentation is mandatory for all plugins and is parsed by tools like ansible-doc. The documentation is defined using static string variables at the top of the plugin's Python file.

Sources: lib/ansible/cli/doc.py:1015-1038, test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:1003-1045

Documentation Blocks

Plugins must include DOCUMENTATION, EXAMPLES, and RETURN blocks. These are YAML strings that provide metadata, usage examples, and return value descriptions.

Block	Purpose
DOCUMENTATION	Contains metadata about the plugin, including short_description, description, version_added, and options.
EXAMPLES	Provides playbook snippets demonstrating how to use the plugin.
RETURN	Describes the values returned by the plugin. This is not applicable to all plugin types but is crucial for lookups.

Important: All Python import statements must appear after these documentation blocks. The validate-modules sanity test will fail with an import-before-documentation error if this rule is violated.

Sources: lib/ansible/modules/lineinfile.py:10-188, test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:1003-1045, 1205-1248

Semantic Markup

Plugin documentation uses a special markup format to create links and apply formatting when rendered by ansible-doc.

Markup	Example	Renders As
B()	B(bold text)	bold text
I()	I(italic text)	italic text (underlined in terminal)
M()	M(ansible.builtin.copy)	[ansible.builtin.copy] (module link)
P()	P(ansible.builtin.file#lookup)	[file] (plugin link)
U()	U(https://ansible.com)	https://ansible.com (URL)
O()	O(path)	'path' (option name)
V()	V(EOF)	'EOF' (option value)

Sources: lib/ansible/cli/doc.py:488-503

Plugin Discovery and Documentation Parsing

The ansible-doc tool is used to view plugin documentation. It discovers, parses, and formats the documentation blocks from plugin files.

The diagram below shows the high-level process for how ansible-doc handles a request for plugin documentation.

Sources: lib/ansible/cli/doc.py:1015-1038, 1102-1132

Jinja2 Built-in Integration

Ansible's documentation system is aware of Jinja2's built-in filters and tests. ansible-doc can display documentation for these built-ins, and Ansible can provide its own documentation that "shadows" the Jinja2 original. This is achieved by dynamically generating a documentation stub for the built-in if no explicit documentation is found within Ansible.

Sources: lib/ansible/cli/doc.py:1062-1065, 1083-1100, test/integration/targets/ansible-doc/test.yml:211-267

Validation and Sanity Testing

Ansible includes a robust sanity testing framework to ensure plugins meet quality and structural standards. The validate-modules test is particularly relevant for plugin developers.

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py

Common Validation Checks

The validate-modules test performs numerous checks, including:

• GPLv3 License Header: The file must start with a valid GPLv3 license header.
• Documentation Presence: DOCUMENTATION and EXAMPLES blocks must exist. RETURN is required for certain plugin types.
• Documentation Syntax: The documentation blocks must be valid YAML.
• Import Order: All import statements must be placed after the documentation blocks.
• Schema Validation: The DOCUMENTATION block must conform to a specific schema, including required fields like short_description.
• Argument Spec Consistency: For modules (and applicable plugins), the argument_spec in the code must match the options documented.

The following diagram outlines the validation process for a plugin file.

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:1719-1784

A failure in these checks will result in a specific error code, which helps developers quickly identify the issue.

Error Code	Description
missing-gplv3-license	The GPLv3 license header was not found in the first 20 lines.
missing-documentation	The DOCUMENTATION block is missing.
documentation-syntax-error	The DOCUMENTATION block is not valid YAML.
import-before-documentation	An import statement was found before the documentation blocks.
invalid-documentation	The documentation does not conform to the required schema (e.g., missing short_description).
undocumented-parameter	An argument is defined in the code but not in the DOCUMENTATION options.

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:1102, 1147, 1213, 1403, 1582, test/integration/targets/ansible-test-sanity-validate-modules/expected.txt:2, 8, 12

Conclusion

Developing Lookup, Filter, and Test plugins is a powerful way to extend Ansible. Success hinges on following established conventions for file structure, class implementation, and, most critically, documentation. The static documentation blocks are not just for human readers; they are a machine-readable API contract parsed and validated by Ansible's tooling. By adhering to these standards, developers can create robust, maintainable, and well-documented plugins that integrate seamlessly into the Ansible ecosystem.

Sources: AGENTS.md:219, lib/ansible/cli/doc.py:476-480