Collections Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• lib/ansible/utils/collection_loader/init.py
• lib/ansible/galaxy/collection/init.py
• test/units/utils/collection_loader/test_collection_loader.py
• test/units/cli/test_galaxy.py
• lib/ansible/galaxy/data/collections_galaxy_meta.yml
• test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/schema.py
• lib/ansible/cli/doc.py
• test/integration/targets/ansible-galaxy-collection/library/setup_collections.py
• test/integration/targets/ansible-galaxy/runme.sh
• test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt
• test/integration/targets/ansible-doc/test.yml

Ansible Collections are the standard format for packaging and distributing Ansible content, including roles, modules, plugins, and documentation. This architecture provides a structured way to manage, share, and consume automation content. The primary tool for interacting with collections is the ansible-galaxy command-line interface.

Collections solve namespace and dependency management challenges by providing a predictable structure and explicit metadata. They allow content to be versioned and distributed independently of the core Ansible engine, facilitating a more modular and scalable automation ecosystem.

Collection Structure

A collection is a directory with a specific structure that Ansible recognizes. The ansible-galaxy collection init command creates a skeleton with the required directories and metadata file.

Sources: test/units/cli/test_galaxy.py:644-655, test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt

The diagram below shows a typical collection directory structure.

Key directories include:

• plugins/: Contains various plugin types like modules, module_utils, filter, lookup, etc.
• roles/: Contains Ansible roles.
• docs/: Holds collection-specific documentation.
• galaxy.yml: The manifest file containing metadata about the collection.

Sources: test/integration/targets/ansible-galaxy-collection-cli/files/expected.txt, test/units/cli/test_galaxy.py:702-706

Collection Metadata (galaxy.yml)

The galaxy.yml file is the manifest for a collection. It defines essential metadata such as the namespace, name, version, dependencies, and authors. This file is mandatory and must be at the root of the collection.

The schema for galaxy.yml defines several keys, some of which are required for a collection to be valid.

Sources: lib/ansible/galaxy/data/collections_galaxy_meta.yml, test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/schema.py:101-110

Key	Required	Type	Description
namespace	Yes	str	The collection's namespace (e.g., company, organization). Must be lowercase alphanumeric with underscores.
name	Yes	str	The name of the collection. Has the same character restrictions as namespace.
version	Yes	str	The collection version, must be a valid semantic version.
readme	Yes	str	Path to the collection's README file, relative to the collection root.
authors	Yes	list	A list of content authors.
description	No	str	A short summary of the collection.
license	No	list	A list of SPDX license identifiers. Mutually exclusive with license_file.
license_file	No	str	Path to the license file. Mutually exclusive with license.
tags	No	list	A list of tags for searching and indexing.
dependencies	No	dict	A dictionary of collection dependencies, with the FQCN as the key and a version specifier as the value.
repository	No	str	URL of the source SCM repository.
documentation	No	str	URL to online documentation.
homepage	No	str	URL to the project's homepage.
issues	No	str	URL to the issue tracker.
build_ignore	No	list	A list of fnmatch patterns to exclude files from the build artifact. Mutually exclusive with manifest.
manifest	No	sentinel	A dict to control MANIFEST.in style directives for building the artifact. Mutually exclusive with build_ignore.

Sources: lib/ansible/galaxy/data/collections_galaxy_meta.yml:10-109

The ansible-galaxy collection CLI

The ansible-galaxy collection command is the primary tool for managing collections. It provides several subcommands to initialize, build, install, and inspect collections.

Sources: test/units/cli/test_galaxy.py:230-233, test/integration/targets/ansible-galaxy/runme.sh:317-320

init

This command creates a new collection skeleton based on a template. It generates the standard directory structure and a default galaxy.yml file.

ansible-galaxy collection init my_namespace.my_collection

The name must be in the format <namespace>.<collection>. The command will fail if the name is invalid.

Sources: test/units/cli/test_galaxy.py:644-655, 820-826

build

This command packages a collection directory into a distributable tarball (.tar.gz). During the build process, it creates two important manifest files inside the archive: MANIFEST.json and FILES.json.

• MANIFEST.json: Contains the collection's metadata from galaxy.yml and information about the FILES.json manifest.
• FILES.json: Lists all files included in the archive along with their checksums and types (file or dir).

The build process can be customized using build_ignore or manifest directives in galaxy.yml to exclude certain files.

Sources: test/units/cli/test_galaxy.py:863-955, lib/ansible/galaxy/data/collections_galaxy_meta.yml:98-109, [test/integration/targets/ansible-galaxy-collection/tasks/build.yml:36-40]

The following diagram illustrates the collection build process.

install

This command installs one or more collections from Ansible Galaxy, a URL, a local .tar.gz file, or a requirements file. It resolves and installs dependencies specified in the galaxy.yml file unless --no-deps is used.

# Install from Galaxy
ansible-galaxy collection install community.general

# Install from a requirements file
ansible-galaxy collection install -r requirements.yml

# Install from a URL
ansible-galaxy collection install https://galaxy.ansible.com/download/community-general-4.0.1.tar.gz

Collections are installed into one of the configured collection paths, defined by ANSIBLE_COLLECTIONS_PATH or the collections_paths setting in ansible.cfg.

Sources: test/units/cli/test_galaxy.py:1020-1038, test/integration/targets/ansible-galaxy/runme.sh:110-115, [test/integration/targets/ansible-galaxy-collection/tasks/download.yml:49-52]

download

This command downloads collection artifacts (.tar.gz) and their dependencies without installing them. It creates a requirements.yml file in the destination directory that can be used to install the downloaded collections later. This is useful for offline or air-gapped environments.

Sources: test/integration/targets/ansible-galaxy-collection/tasks/download.yml:5-10, 29-34

list

This command lists all installed collections, showing their name, version, and the path where they are installed. It can also filter the list by collection name.

Sources: test/integration/targets/ansible-galaxy/runme.sh:388-393, [lib/ansible/cli/doc.py:839-843]

Collection Loading

Ansible's plugin loader is responsible for finding and loading content (modules, roles, etc.) from installed collections. It searches for collections in the paths specified by the ANSIBLE_COLLECTIONS_PATH environment variable or the collections_paths configuration setting.

When a playbook references content using a Fully Qualified Collection Name (FQCN), like my_namespace.my_collection.my_module, the loader searches the configured paths for the my_namespace/my_collection directory and then locates the content within it.

The search path for collections can be viewed using ansible-doc. The loader also handles collection-based documentation loading for tools like ansible-doc.

Sources: lib/ansible/cli/doc.py:839-843, test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/main.py:53-65, [test/integration/targets/ansible-galaxy/runme.sh:386-387]

Validation

The system includes robust validation for collection metadata and structure. The validate-modules sanity test, for example, uses schemas to enforce correctness in galaxy.yml and documentation strings. This includes:

• Validating FQCNs for modules and plugins.
• Ensuring version_added and deprecation versions follow semantic versioning.
• Checking for required fields like namespace, name, and version.
• Validating data types and structures for all metadata fields.

Sources: test/lib/ansible_test/_util/controller/sanity/validate-modules/validate_modules/schema.py:101-110, 313-333, 1079-1100

Conclusion

The Ansible Collections architecture provides a standardized, robust, and scalable system for managing Ansible content. By enforcing a clear structure via galaxy.yml and providing a powerful management tool in ansible-galaxy, collections enable developers and users to create, share, and consume automation content efficiently and reliably. The built-in mechanisms for dependency management, versioning, and namespacing are central to preventing conflicts and ensuring playbook portability across different environments.