Testing Strategy & Pytest Fixtures

Relevant source files

The following files were used as context for generating this wiki page:

• examples/basic-plugin.md
• examples/README.md
• tests/unit/test_plugin.py
• tests/unit/test_discovery.py
• tests/unit/test_examples.py
• pyproject.toml

The testing strategy for the FLEXT Plugin system is built on a multi-layered approach designed to ensure technical accuracy, performance, and reliability. It spans isolated unit tests for core domain entities, integration tests for the plugin platform, and automated execution of example scripts to verify real-world usability. The system leverages pytest as its primary testing framework, utilizing specialized fixtures and markers to manage plugin lifecycles and environmental dependencies.

Sources: examples/basic-plugin.md:394-406, pyproject.toml:344-370

Testing Hierarchy and Categories

The project categorizes tests into several tiers to maintain high quality standards. According to project guidelines, examples must maintain at least 85% test coverage.

Test Classifications

Tests are organized by their scope and execution requirements using pytest markers defined in the project configuration:

Marker	Description
unit	Isolated tests for individual methods and domain entities.
integration	Tests verifying the interaction between plugins and the platform.
docker	Tests requiring active Docker services (e.g., PostgreSQL, Redis).
performance	Benchmarks for critical paths like execution time.
e2e	End-to-end integration tests for complete workflows.
slow	Long-running tests excluded from quick validation cycles.

Sources: pyproject.toml:356-368, examples/README.md:309-315

Testing Workflow

The testing workflow follows a strict validation chain, ensuring that root causes are fixed and validated in a single turn.

The diagram shows the sequential validation of components from core logic to external integrations.
Sources: tests/unit/test_examples.py:16-20, pyproject.toml:356-368

Pytest Fixtures

Fixtures are used to manage the lifecycle of plugins and the platform during testing, providing clean state isolation for every test case.

Core Platform Fixtures

The most critical fixtures manage the FlextPluginPlatform and the PluginRegistry to ensure that registrations and deactivations do not leak between tests.

@pytest.fixture
def platform():
    """Create test platform fixture with test_mode enabled."""
    platform = create_flext_plugin_platform(settings={"test_mode": True})
    yield platform
    platform.shutdown()

@pytest.fixture(autouse=True)
def reset_registry(self) -> None:
    """Reset class-level storage before each test."""
    registry = FlextPluginPlatform.PluginRegistry.create()
    plugins_result = registry.list_plugins()
    if plugins_result.success:
        for plugin_name in plugins_result.value:
            _ = registry.unregister(plugin_name)

Sources: examples/README.md:63-69, tests/unit/test_plugin.py:104-111

Data and Resource Fixtures

Fixtures are also used to provide standard configurations and temporary file system resources for discovery testing.

Fixture	Purpose	Source File
plugin_config	Returns a dictionary with default batch sizes and timeouts.	examples/basic-plugin.md
temp_dir	Provides a Path to a temporary directory for discovery tests.	tests/unit/test_discovery.py
discovery	Initializes a FlextPluginDiscovery instance.	tests/unit/test_discovery.py
plugin	Creates a pre-configured plugin instance (e.g., BasicDataProcessorPlugin).	examples/basic-plugin.md

Sources: examples/basic-plugin.md:278-292, tests/unit/test_discovery.py:30-43

Lifecycle Testing Strategy

The testing strategy emphasizes verifying the plugin lifecycle, from discovery and initialization to execution and cleanup.

Sequence of Lifecycle Validation

Tests verify that plugins transition correctly through statuses such as INACTIVE, ACTIVE, and ERROR.

The diagram illustrates the standard interaction pattern between the test suite, the platform, and a plugin.
Sources: examples/basic-plugin.md:347-380, tests/unit/test_plugin.py:123-146

Error Handling and Resilience

Testing includes forcing failures to verify the "r pattern" (result pattern) usage. This ensures that when a plugin fails, it returns a failed result instead of raising unhandled exceptions. Tests specifically verify:

• Execution without initialization.
• Execution while inactive.
• Input data validation failures.
• Internal processing errors caught via mocks.

Sources: examples/basic-plugin.md:316-345, examples/README.md:255-265

Automated Example Verification

A unique aspect of the testing strategy is the verification of example scripts. The TestsFlextPluginExamples suite uses subprocess to run .py files in the examples/ directory to ensure they remain functional as the core API evolves.

Test Method	Target Example	Logic
test_basic_plugin_example_execution	01_basic_plugin.py	Runs via sys.executable; checks exit code 0.
test_docker_integration_example_execution	03_docker_integration.py	Runs with run argument; validates plugin creation.
test_docker_integration_example_with_connection_testing	03_docker_integration.py	Uses --test-connections; marked to skip without Docker.

Sources: tests/unit/test_examples.py:25-66, examples/03_docker_integration.py:126-150

Performance and Quality Gates

The testing strategy is enforced by "Quality Gates" defined in the project configuration.

• Coverage: The pyproject.toml configuration sets a fail_under threshold for coverage reports.
• Benchmarks: Performance tests measure execution time, ensuring that data processing completes within defined limits (e.g., < 1.0s for standard payloads).
• Static Analysis: The suite is complemented by mypy for type checking and ruff for linting, both of which are integrated into the overall make check and make test verbs.

Sources: pyproject.toml:137-142, examples/basic-plugin.md:412-436, README.md:36-39