Documentation Generation Pipeline
Relevant source files
The following files were used as context for generating this wiki page:
The Documentation Generation Pipeline automates the creation of plugin reference pages from Python source code to maintain a single source of truth. It extracts YAML docstrings from Ansible plugins and transforms them into MDX files for the Docusaurus-based documentation site. This system prevents documentation drift by ensuring the reference site remains identical to the validated interfaces enforced by ansible-test sanity.
Sources: dev/adr/0005-doc-generation-pipeline.md:10-18, dev/guidelines/documentation.md:12-19
Pipeline Architecture
The pipeline follows a structured data flow from raw Python docstrings through parsing and templating to final static site assets.
The diagram shows the transformation of plugin Python files into MDX artifacts via Invoke tasks and Jinja2 templates.
Sources: tasks/docs.py:209-281, dev/guidelines/documentation.md:12-19
Core Components and Logic
Plugin Data Extraction
The tasks/docs.py module contains the logic to scan and parse plugin files. The find_plugin_files function identifies Python files in plugins/modules/, plugins/inventory/, and plugins/lookup/, excluding __init__.py files.
Sources: tasks/docs.py:46-63
The extract_docstring function retrieves specific module-level variables:
DOCUMENTATION: The module's identity and option schema.EXAMPLES: Playbook snippets for usage reference.RETURN: The structure of returned data.
Sources: tasks/docs.py:66-93, dev/guidelines/module-docstrings.md:10-21
MDX Safety and Formatting
Because Docusaurus treats curly braces {} as JSX expressions, the pipeline includes an mdx_safe filter. This function escapes braces in prose while preserving Markdown code spans (backticks) to prevent rendering errors.
Sources: tasks/docs.py:22-38, dev/adr/0005-doc-generation-pipeline.md:27-30
Template Engine
The pipeline uses Jinja2 templates located in docs/_templates/ to define the layout of the generated pages.
| Template | Output Destination | Purpose |
|---|---|---|
plugin.mdx.j2 | docs/docs/references/plugins/*.mdx | Generates individual plugin reference pages. |
readme.mdx.j2 | docs/docs/readme.mdx | Generates the landing page with a plugin index. |
Sources: dev/guidelines/documentation.md:21-36, tasks/docs.py:228-232
Documentation Sources and Fragments
Single Source of Truth
Developers MUST update the docstrings in the Python files rather than the generated MDX files. The pipeline overwrites generated artifacts every time it runs.
Sources: dev/adr/0005-doc-generation-pipeline.md:33-37, dev/guidelines/module-docstrings.md:32-37
Doc Fragments
To avoid duplication, shared options (such as api_endpoint, token, and timeout) are stored as fragments in plugins/doc_fragments/fragments.py. Plugins pull these shared definitions using the extends_documentation_fragment key within their DOCUMENTATION blocks.
Sources: dev/guidelines/module-docstrings.md:23-30, dev/constitution.md:25-27
Operational Commands
The pipeline is executed using invoke tasks.
| Command | Action |
|---|---|
invoke generate-doc | Scans plugins, extracts docstrings, and renders MDX files via Jinja2. |
invoke docusaurus | Runs npm run build inside the docs/ directory to generate the static site. |
vale docs/ | Performs prose linting on the documentation site for style consistency. |
Sources: dev/guidelines/documentation.md:12-19, AGENTS.md:31-41, tasks/docs.py:274-284
Implementation Constraints
- Active Voice: Documentation guidelines require neutral, subject-first sentences and active voice verbs. Sources: dev/guidelines/documentation.md:105-115
- Validation: Every change to a module docstring MUST be followed by
invoke generate-docin the same change to ensure accuracy. Sources: dev/constitution.md:73-77 - JSX Escaping: The pipeline MUST handle JSX-reserved braces to prevent build failures in Docusaurus. Sources: dev/adr/0005-doc-generation-pipeline.md:27-30
The Documentation Generation Pipeline ensures that technical documentation remains technically accurate, structurally consistent, and automatically synchronized with the codebase.