Example Workflows & Executable Docs
Relevant source files
The following files were used as context for generating this wiki page:
Example Workflows and Executable Docs within the FLEXT ecosystem serve as foundational blueprints for enterprise data integration. These examples demonstrate advanced capabilities such as parallel batch processing, intelligent server auto-detection, and functional error handling using the Railway Pattern. They are designed to be "executable documentation," meaning they provide verified, live code paths that validate architecture patterns while serving as educational resources for developers.
The scope of these workflows covers end-to-end integration cycles, ranging from Access Control List (ACL) processing and complex data validation to full-scale workflow orchestration. By adhering to the Universal Agent Law defined in the project governance, these examples ensure that tests and snippets remain valid even when underlying configurations change, preventing "frozen" scalar values in test suites.
Sources: examples/README.md:16-56, AGENTS.md:15-28
Core Architecture Patterns
The workflows are built upon three primary architectural pillars: the Railway Pattern, Parallel Processing, and Type Safety.
The Railway Pattern
All examples implement a failure-resistant pipeline following functional programming principles. Operations return a result type (r[T]), allowing the pipeline to stop on the first failure without exception propagation. This pattern ensures comprehensive error reporting and context tracking throughout the data lifecycle.
Sources: examples/README.md:60-64, examples/complete_workflow_example.py:167-170
Parallel Execution Logic
High-performance processing is achieved using ThreadPoolExecutor. The system supports configurable worker thread pools and batch processing to prevent memory exhaustion during heavy operations.
The diagram shows the decision logic between parallel and sequential execution based on context configuration.
Sources: examples/advanced_processing_example.py:221-240, examples/README.md:66-70
Workflow Orchestration Structure
| Component | Description | Reference Class |
|---|---|---|
| Orchestrator | Manages resource lifecycle and stage sequencing. | WorkflowOrchestrator |
| Context | Carries correlation IDs, start times, and metadata. | WorkflowContext |
| Stage Result | Tracks success/failure, timing, and items per stage. | WorkflowStageResult |
| Payload | Typed container for data passing between stages. | PipelineStageData |
Sources: examples/complete_workflow_example.py:44-123, examples/advanced_processing_example.py:64-80
Implementation Modules
1. Advanced Processing Pipeline
The AdvancedProcessingExample demonstrates a declarative pipeline where stages such as validate, process, and analyze are executed sequentially on a data set.
Key Components:
_validate_batch: Performs parallel item checking for missing fields or data violations._process_parallel: Uses a thread pool to apply transformations to items concurrently._analyze_results: Aggregates metadata, including field distribution and complexity scores.
Sources: examples/advanced_processing_example.py:126-218, examples/README.md:38-48
2. Complete Workflow Orchestration
The CompleteWorkflowExample integrates all capabilities into a unified execution model. It features a correlation tracking system and an automatic resource management lifecycle.
This diagram illustrates the lifecycle of a complete workflow, highlighting the separation between orchestration and parallel execution.
Sources: examples/complete_workflow_example.py:125-276
3. ACL and Server Integration
Workflows often involve external integrations, such as Oracle WMS or LDAP servers. The acl_processing_example.py (referenced in documentation) specifically targets:
- Server Auto-detection: Detecting LDAP types (Active Directory, OID, OUD).
- Forbidden Combinations: Validating complex ACL rules per server type.
- Throughput Analysis: Measuring ACL extraction performance.
Sources: examples/README.md:21-36, docker/docker-compose.oracle-wms.yml:10-40
Executable Documentation Standards
Per AGENTS.md, examples are not just static code; they are "Executable Documentation" that must remain valid under all conditions.
- P0 Governance: Examples and docstring snippets must not hardcode scalar values found in
config/*.yaml. They must read from the same Typed Single Source of Truth (SSOT) as production code. - Validation Rigor: The
examples/directory is treated as first-class code. It is subject toruff,pyrefly,pyright, andmypygates with the same strictness as thesrc/directory.
Sources: AGENTS.md:15-28, AGENTS.md:52-57
Performance Monitoring
Workflows capture detailed metrics to ensure enterprise-grade observability.
| Metric | Calculation / Source | Purpose |
|---|---|---|
workflow_efficiency | Succeeded Items / Processed Items | Measures data quality and pipeline health. |
parallel_utilization | Total CPU Time / Real Time | Measures thread pool efficiency. |
workflow_throughput | Total Items / Total Time | Measures system performance capacity. |
avg_complexity | (Field count * 0.1) / Items | Estimates computational weight per item. |
Sources: examples/complete_workflow_example.py:186-200, examples/advanced_processing_example.py:170-185
Conclusion
Example Workflows & Executable Docs provide a robust framework for building and validating enterprise integrations. By combining the Railway Pattern for reliability and ThreadPoolExecutor for performance, these modules serve as both functional tools and living documentation. Adherence to strict typing and project governance ensures that these examples remain maintainable and technically accurate as the FLEXT ecosystem evolves.
Sources: examples/README.md:108-115, AGENTS.md:104-110