Result Flow & Returns

Relevant source files

The following files were used as context for generating this wiki page:

• AGENTS.md
• examples/advanced_processing_example.py
• examples/complete_workflow_example.py
• examples/README.md
• pyproject.toml

Result Flow and Returns in the FLEXT platform are governed by a strict "Railway Pattern" architecture. This system ensures that every fallible application path returns a typed r[T] container instead of raising raw exceptions or returning untyped error dictionaries. This approach centralizes error handling, enforces type safety, and facilitates complex pipeline orchestration across enterprise data integration tasks.

The system relies on functional programming principles where operations either stay on the "success track" (returning data) or shunt to the "failure track" (returning error context). This pattern is mandatory for all flext-* packages and is validated through static analysis tools like MyPy and Pyright to ensure zero Any bypass.

Sources: AGENTS.md:387-393, examples/README.md:65-71

Railway Pattern Architecture

The Railway Pattern is the foundation of the FLEXT return system. It utilizes the r[T] result container (often aliased as p.Result or simply r) to encapsulate both successful values and failure states.

Core Components

• Result Container (r[T]): A wrapper that holds either a value of type T or an error. Sources: AGENTS.md:511-512
• Success Track: Operations that succeed return r.ok(value). Sources: examples/complete_workflow_example.py:168
• Failure Track: Operations that encounter issues return r.fail(message). Sources: examples/advanced_processing_example.py:126
• Early Termination: In a pipeline, the first encountered failure halts subsequent stages, preventing cascading errors. Sources: examples/README.md:70

The following flowchart illustrates the logical progression of the Railway Pattern within a FLEXT pipeline:

This diagram shows how a failure at any stage immediately diverts the execution flow to the failure track.

Sources: examples/README.md:65-75, examples/advanced_processing_example.py:128-135

Pipeline Execution & Stage Returns

FLEXT utilizes declarative processing pipelines where results from one stage serve as the input for the next. Each stage is an isolated function that consumes a payload and returns a Result container.

Pipeline Orchestration

A typical pipeline involves three primary stages:

1. Validation: Checks item integrity and permissions.
2. Processing: Executes the core business logic (often in parallel).
3. Analysis: Aggregates results and generates performance metrics.

Component	Functionality	Return Type
_validate_batch	Validates a set of items before processing.	p.Result[PipelineStageData]
_process_parallel	Executes item processing using ThreadPoolExecutor.	p.Result[PipelineStageData]
_analyze_results	Provides data insights and calculates efficiency.	p.Result[PipelineStageData]

Sources: examples/advanced_processing_example.py:128-132, examples/README.md:34-40

Multi-Stage Flow Example

In the FlextLdifProcessingPipeline, the orchestrator iterates through defined stages. If any stage returns a failure, the entire pipeline returns that failure result immediately.

This sequence shows the synchronous progression of stages where each stage must return a successful Result container to proceed.

Sources: examples/advanced_processing_example.py:128-142

Parallel Result Aggregation

When processing items in parallel, FLEXT uses ThreadPoolExecutor but wraps the outcomes back into the standardized Result flow. This ensures that even asynchronous operations maintain consistent return contracts.

Handling Parallel Outcomes

The orchestrator manages individual item results and aggregates them into a collective stage result. This includes tracking:

• Items Succeeded: Count of items that returned valid results.
• Items Failed: Count of items that encountered exceptions or validation errors.
• Processing Time: Duration of the parallel execution block.

Sources: examples/complete_workflow_example.py:228-260

Workflow Result Structure

The CompleteWorkflowResult model provides a holistic view of an entire multi-stage operation.

Field	Type	Description
workflow_id	str	Unique identifier for the workflow run.
stage_results	SequenceOf[WorkflowStageResult]	List of results from each executed stage.
total_processing_time	float	Cumulative time across all stages.
aggregated_metrics	JsonMapping	Efficiency, throughput, and utilization data.

Sources: examples/complete_workflow_example.py:96-120

Implementation Constraints

FLEXT architecture imposes several non-negotiable constraints on result handling to maintain enterprise-grade reliability.

• No Raw Exceptions: Custom error flow must use r[T]. Raw exceptions are reserved for truly unrecoverable system failures, not logic branching. Sources: AGENTS.md:511-512
• Strict Typing: Every result must be typed. The use of Any or bare object is prohibited and caught by MyPy via the returns_plugin. Sources: pyproject.toml:169, AGENTS.md:465
• Correlation Tracking: Results must include correlation IDs and metadata for observability in distributed environments. Sources: examples/complete_workflow_example.py:101, examples/README.md:120

# Standard FLEXT Result Handling Example
def execute(self) -> p.Result[PipelineStageData]:
    for operation in operations:
        result = operation(current_data)
        if result.failure:
            return result  # Early termination on failure
        current_data = result.value.data.values
    return r[PipelineStageData].ok(PipelineStageData(data=payload))

Sources: examples/advanced_processing_example.py:133-142

Conclusion

Result Flow & Returns in FLEXT are designed for predictability and resilience. By enforcing the Railway Pattern and standardized Result containers, the platform eliminates the ambiguity of return values and the instability of unmanaged exceptions. This structured approach allows developers to build complex, parallelized data integration workflows that are self-documenting through their types and robust in the face of partial failures.

Sources: examples/README.md:128-135, AGENTS.md:387-393