Testing Strategy Overview

Relevant source files

The following files were used as context for generating this wiki page:

• pyproject.toml
• tests/copier_template/test_generated.py
• tests/test_meltano_edk.py
• tests/test_process.py
• meltano/edk/extension.py

The Meltano Extension Developer Kit (EDK) employs a multi-layered testing strategy designed to ensure the reliability of the framework itself and the validity of the extensions it generates. The strategy encompasses unit testing of core logic, mocking of asynchronous subprocess execution, and automated validation of the extension template through static analysis and formatting checks.

Key testing dependencies include pytest for the test runner, mypy for type checking, and ruff for linting and formatting. The infrastructure supports testing both synchronous extension interfaces and asynchronous process management.
Sources: pyproject.toml:60-65, tests/copier_template/test_generated.py:12-25

Core Framework Testing

Testing of the core EDK framework focuses on the lifecycle of a Meltano extension. This includes verifying the behavior of the ExtensionBase class, specifically the orchestration of pre-invocation, invocation, and post-invocation steps.

Extension Lifecycle Validation

The framework tests use a CustomExtension implementation to track the execution order and arguments of various lifecycle methods. This ensures that the pass_through_invoker correctly triggers the expected sequence of events.

Formatted Output Testing

The EDK supports multiple output formats for extension descriptions (text, JSON, YAML). Tests verify that the describe_formatted method correctly serializes the Describe model into these formats.

Format	Library Used	Validation Method
JSON	json	json.loads() and dict comparison
YAML	PyYAML	yaml.safe_load() and dict comparison
Text	devtools	pformat() check

Sources: tests/test_meltano_edk.py:46-85, meltano/edk/extension.py:86-100

Subprocess and Async Testing

A critical component of the EDK is the Invoker class, which manages external process execution. Testing this module requires mocking asyncio subprocesses to simulate standard output (stdout) and standard error (stderr) streams without executing real binaries.

Mocking Strategy

The test suite utilizes unittest.mock.AsyncMock and patch to intercept asyncio.create_subprocess_exec. This allows for simulating process behaviors such as:

• EOF (End Of File) signals on streams.
• Line-by-line reading of simulated process output.
• Verification of environment variables and working directory (CWD) propagation.

Template Generation Testing

The EDK provides a copier template for generating new extensions. The testing strategy includes a specific suite to ensure that generated code is syntactically correct and adheres to project standards.

Integration Validation Workflow

1. Generation: The copier.run_copy function generates a new project in a temporary directory using a standard set of data (e.g., extension_id, wrapper_target_name).
2. Type Checking: mypy is run against the generated output to ensure type safety.
3. Linting: ruff check is executed to verify adherence to coding standards (specifically codes E, F, W, and Q).
4. Formatting: ruff format --check ensures the generated code matches the expected style guide.

Sources: tests/copier_template/test_generated.py:32-75, copier.yml:1-35

Dependency Groups and Tools

The testing infrastructure is defined in the project configuration, ensuring consistent environments across developer machines and CI/CD pipelines.

Tool	Purpose	Configuration File
pytest	Main test runner	pyproject.toml
mypy	Static type analysis	pyproject.toml
ruff	Linting and Formatting	pyproject.toml
structlog	Structured logging validation	meltano/edk/logging.py

Sources: pyproject.toml:57-66, meltano/edk/logging.py:89-130

Summary

The Meltano EDK testing strategy combines standard Python unit testing with specialized mocks for asynchronous subprocess management and template validation. By integrating static analysis tools like mypy and ruff directly into the template testing workflow, the framework ensures that both the EDK itself and the extensions produced by users maintain high code quality and architectural consistency.