Plugin Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• lib/ansible/cli/doc.py
• lib/ansible/utils/plugin_docs.py
• lib/ansible/plugins/list.py
• test/lib/ansible_test/_util/controller/sanity/validate-modules/main.py
• test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt
• lib/ansible/plugins/doc_fragments/action_core.py
• README.md

Ansible's plugin architecture is a cornerstone of its design, providing a flexible and powerful system for extending its core functionality. It allows for the development and integration of various components such as modules, filters, lookups, and more. This system is designed to be simple, agentless, and human-friendly, aligning with Ansible's core design principles. The modern approach to distributing plugins is through Ansible Collections, which package content for easy sharing and consumption.

This document details the internal mechanics of Ansible's plugin system, including how plugins are discovered, documented, and validated, based on an analysis of the source code.

Core Concepts

Ansible's functionality is extended through a variety of plugin types. The ansible-doc tool can display information for a specific set of these types.

Documentable Plugin Types

The following plugin types are considered "documentable" and can be inspected using ansible-doc -t <type>.

Plugin Type	Description
module	The primary unit of work in Ansible, executed on target nodes.
action	Executes on the controller, often in conjunction with a module.
become	Manages privilege escalation (e.g., sudo, su).
cache	Caches facts and other data.
callback	Adds hooks to respond to Ansible events.
cliconf	Manages connections and command execution on network devices.
connection	Manages the connection to the target host (e.g., ssh, docker).
doc_fragment	Reusable snippets of documentation for other plugins.
filter	Manipulates data within templates.
httpapi	Provides an interface to web services.
inventory	Parses inventory sources.
lookup	Retrieves data from external sources.
netconf	Manages NETCONF connections for network devices.
shell	Defines target host shell environments.
strategy	Controls the flow of playbook execution.
terminal	Manages terminal interaction for network devices.
test	Jinja2 tests to verify data.
vars	Injects variables into a run.
role	A special type for documenting role argument_specs.
keyword	A special type for documenting playbook keywords.

Sources: lib/ansible/cli/doc.py:65-66

Plugin Discovery and Loading

Ansible uses a sophisticated search mechanism to discover and load plugins from both legacy paths and Ansible Collections. The ansible-doc CLI tool provides a clear example of this process.

The diagram below illustrates the high-level process of how Ansible discovers plugins.

Sources: lib/ansible/plugins/list.py:183-214

Key Components

• Plugin Loaders: Specific loaders exist for each plugin type (e.g., module_loader, filter_loader). These are accessible via ansible.plugins.loader. The DocCLI in ansible-doc prepares the appropriate loader using the _prep_loader static method.
• Search Paths: Loaders search for plugins in a defined set of paths, which includes directories within configured collections, legacy paths like ANSIBLE_LIBRARY, and paths relative to the current playbook.
• Collections: The primary mechanism for distributing plugins. Ansible searches for collections in configured paths (ANSIBLE_COLLECTIONS_PATH). The list_collections function is used to find all available collections.
• _list_plugins_with_info: This function orchestrates the discovery process. It determines which collections to search (all, or a specified subset) and calls _list_collection_plugins_with_info to gather plugins from each.
• _list_collection_plugins_with_info: For each collection, this function identifies the correct plugin directories (e.g., plugins/modules/, plugins/filter/) and scans them for plugin files. It handles special logic for ansible.builtin and ansible.legacy collections.

Sources: lib/ansible/cli/doc.py:1005-1021, lib/ansible/plugins/list.py:53-214

Plugin and Collection Structure

Plugins are organized into a standard directory structure, especially within collections. This structure allows Ansible to automatically discover and load content.

The following diagram shows a typical directory structure for an Ansible Collection.

Sources: test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt

• plugins/: The root directory for all plugin types within a collection.
• <plugin_type>/: Each subdirectory under plugins/ corresponds to a specific plugin type (e.g., modules, filter, lookup).
• Plugin Files: Python files (.py) within these directories are treated as plugins. For some types, other file extensions like .ps1 (PowerShell) or .yml (for sidecar documentation) are also recognized.

Sources: test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt, test/lib/ansible_test/_util/controller/sanity/validate-modules/main.py:151-159

Plugin Documentation System

A key feature of the plugin architecture is its self-documenting nature. Plugin authors embed documentation directly within the plugin files as specially named string variables. The ansible-doc tool then parses these variables to generate documentation.

Documentation Sections

Section	Variable Name	Purpose
Documentation	DOCUMENTATION	A YAML string containing metadata, descriptions, and option definitions.
Examples	EXAMPLES	A YAML or multi-line string containing usage examples.
Return Values	RETURN	A YAML string describing the data returned by the plugin.

Sources: lib/ansible/cli/doc.py:1669-1678, test/lib/ansible_test/_util/controller/sanity/validate-modules/main.py:809-847

Documentation Generation Flow

The following diagram shows the sequence of events when ansible-doc is used to retrieve documentation for a plugin.

Sources: lib/ansible/cli/doc.py:1059-1070, lib/ansible/utils/plugin_docs.py:279-318

Documentation Fragments

To promote reuse, documentation can be shared across multiple plugins using "documentation fragments".

• A plugin specifies which fragments to include via the extends_documentation_fragment key in its DOCUMENTATION block.
• The add_fragments function in lib/ansible/utils/plugin_docs.py is responsible for finding and merging the content from the specified doc_fragment plugins.
• Fragments themselves are plugins of type doc_fragment and contain their own DOCUMENTATION or RETURN sections.

Sources: lib/ansible/utils/plugin_docs.py:127-194, lib/ansible/plugins/doc_fragments/action_core.py

Sidecar Documentation

For plugins written in languages other than Python (like PowerShell), or to separate documentation from code, a "sidecar" documentation file can be used. This is a YAML file with the same base name as the plugin file, located in the same directory (e.g., my_module.ps1 and my_module.yml). ansible-doc will automatically look for and parse this file.

Sources: lib/ansible/utils/plugin_docs.py:293-300, test/lib/ansible_test/_util/controller/sanity/validate-modules/main.py:800-818

Role Argument Specification

Roles can also be documented using a similar mechanism, specifically for their input parameters. This is handled by the RoleMixin within the DocCLI class.

• Argument Spec Files: ansible-doc looks for role argument specifications in the role's meta/ directory.
• Precedence: It checks for files in a specific order, with argument_specs.yml (and its variants) taking precedence over main.yml.

File (in meta/ directory)	Precedence
argument_specs.yml / .yaml	1 (Highest)
main.yml / .yaml	2 (Lowest)

• Loading: The _load_argspec method loads the YAML content and extracts the data under the argument_specs top-level key.

Sources: lib/ansible/cli/docpy:174-175, 203-220

Plugin Validation

Ansible includes a robust validation system, executed via the ansible-test sanity command, to ensure plugins adhere to architectural and style guidelines. The validate-modules test is a key part of this.

The ModuleValidator class performs numerous checks, including:

• Licensing: Verifies the presence of a GPLv3 header.
• Interpreter Shebang: Ensures Python and PowerShell modules have the correct #! line.
• Import Order: Checks that imports appear after the DOCUMENTATION, EXAMPLES, and RETURN blocks.
• Schema Validation: The DOCUMENTATION and RETURN sections are validated against a strict schema to ensure correctness and consistency.
• Argument Spec Consistency: Compares the argument_spec defined in the code with the options documented in the DOCUMENTATION block, checking for mismatches in types, defaults, and required status.
• Semantic Markup: Validates the correctness of special markup used in descriptions, such as M(), U(), and O().

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/main.py:408-415, 608-638, 850-891, 1073-1323, test/integration/targets/ansible-test-sanity-validate-modules/expected.txt.

Summary

Ansible's plugin architecture is a comprehensive system that defines not just how to extend functionality but also how that functionality is discovered, documented, and validated. It relies on convention over configuration, with standardized directory structures and embedded documentation. The shift towards collections has further modularized this system, making it easier for the community to create and share Ansible content. Understanding this architecture is crucial for any developer looking to contribute to or extend the Ansible ecosystem.