Workflow Execution Engine

Relevant source files

The following files were used as context for generating this wiki page:

• backend/src/workflows/application/execution_service.py
• backend/src/workflows/domain/execution_entities.py
• backend/src/workflows/application/use_cases/execute_workflow.py
• backend/src/workflows/application/action_executor.py
• backend/src/workflows/application/retry_service.py
• backend/src/workflows/domain/execution_exceptions.py
• backend/src/workflows/infrastructure/execution_models.py

The Workflow Execution Engine is the core automation component responsible for managing the lifecycle and runtime execution of workflows. It orchestrates the transition between different execution states, manages sequential and conditional action processing, and ensures resilience through specialized retry logic and error categorization.

This system is built using a Domain-Driven Design (DDD) approach, separating the core state machine logic (Domain Layer) from the execution orchestration (Application Layer) and persistence mechanisms (Infrastructure Layer). It facilitates complex automation scenarios including communication (Email/SMS), CRM updates, and external integrations via webhooks.

Sources: DDD_IMPLEMENTATION_REPORT_SPEC_WFL_005.md, specs/workflows/plan.md:SPEC-WFL-005

Architecture and Components

The engine's architecture is divided into specialized services and entities that handle different aspects of the execution lifecycle.

Core Components

Component	Layer	Description
WorkflowExecution	Domain	Aggregate root representing the state and progress of a single contact's journey through a workflow.
WorkflowExecutionService	Application	The primary orchestrator that processes steps, evaluates conditions, and invokes executors.
ActionExecutor	Application	A factory-driven interface for executing specific action types (e.g., SendEmail, Webhook).
RetryService	Application	Manages exponential backoff and error-based retry strategies for failed actions.
ExecutionLog	Domain	An audit trail entity capturing the details, duration, and results of every executed step.

Sources: DDD_IMPLEMENTATION_REPORT_SPEC_WFL_005.md, backend/src/workflows/application/execution_service.py

Component Relationship Diagram

The following diagram illustrates the high-level relationship between the execution use cases, the core service, and the supporting infrastructure.

Execution Lifecycle and State Machine

The engine employs a 7-state lifecycle machine to track the status of a workflow execution. Transitions are strictly validated to prevent illegal state changes (e.g., moving from COMPLETED to ACTIVE).

Execution States

• QUEUED: Initial state when a trigger creates an execution.
• ACTIVE: The execution is currently being processed by the engine.
• PAUSED: Execution manually halted; can be resumed.
• WAITING: Execution suspended due to a "Wait" action step.
• COMPLETED: All steps finished successfully or a goal was achieved.
• FAILED: Execution stopped due to an unrecoverable error or exhausted retries.
• CANCELLED: Manually terminated by a user or system event.

Sources: backend/src/workflows/domain/execution_entities.py, DDD_IMPLEMENTATION_REPORT_SPEC_WFL_005.md

State Transition Flow

This flowchart describes the valid transitions within the execution state machine.

Action Execution and Retry Logic

Actions are processed sequentially. The WorkflowExecutionService identifies the current step, retrieves the appropriate executor via the ActionExecutorFactory, and attempts execution.

Resilience Strategy

The RetryService provides an exponential backoff strategy with jitter to handle transient failures, such as network timeouts or rate limits (429 errors).

Error Category	Retryable	Strategy
timeout	Yes	Exponential backoff
rate_limit	Yes	Backoff based on retry headers
server_error	Yes	Exponential backoff
validation	No	Immediate failure/skip
authentication	No	Immediate failure

Sources: backend/src/workflows/application/retry_service.py, DDD_IMPLEMENTATION_REPORT_SPEC_WFL_005.md

Action Processing Sequence

The following sequence diagram shows the interaction between the service, executors, and the retry mechanism.

Infrastructure and Data Model

The execution state and audit logs are persisted in PostgreSQL using SQLAlchemy models.

Data Model Fields

Model	Field	Type	Description
WorkflowExecutionModel	status	Enum	Current lifecycle state (ACTIVE, FAILED, etc.)
	current_step_index	Integer	Index of the last successful or currently running step.
	retry_count	Integer	Number of times the current action has been retried.
ExecutionLogModel	duration_ms	Integer	Time taken for the action to execute.
	error_details	JSON	Captures stack traces or API error responses.
	response_data	JSON	Captures successful response payloads from actions.

Sources: backend/src/workflows/infrastructure/execution_models.py, DDD_IMPLEMENTATION_REPORT_SPEC_WFL_005.md

Specialized Constraints and Compliance

The engine enforces specific business rules and legal compliance requirements during execution:

1. Opt-out Prevention: The ContactOptedOutError is raised if a contact has unsubscribed, blocking communication actions (REQ-N1).
2. Concurrent Limits: A default limit (e.g., 100/account) is enforced to prevent resource exhaustion (REQ-N2).
3. Data Security: Sensitive action configurations (like API keys in webhooks) are encrypted within the persistence layer.

Sources: backend/src/workflows/domain/execution_exceptions.py, DDD_IMPLEMENTATION_REPORT_SPEC_WFL_005.md

Summary

The Workflow Execution Engine provides a robust, state-aware environment for automation. By leveraging a centralized state machine and specialized executors, it ensures that contact journeys are processed reliably, errors are handled gracefully with retries, and all activities are logged for auditing and analytics purposes.