Workflow Execution Engine

Relevant source files

The following files were used as context for generating this wiki page:

• packages/core/src/execution-engine/workflow-execute.ts
• packages/core/src/execution-engine/routing-node.ts
• packages/cli/src/workflow-runner.ts
• packages/core/src/execution-engine/execution-lifecycle-hooks.ts
• packages/core/src/execution-engine/tests/workflow-execute.test.ts
• packages/nodes-base/nodes/Github/Github.node.ts

The Workflow Execution Engine is the core component responsible for running n8n workflows. It manages the entire lifecycle of a workflow execution, from initiation by a trigger or manual start, through the processing of each node, to the final completion or failure. The engine is encapsulated within the WorkflowExecute class, which orchestrates data flow, state management, error handling, and interaction with various node types.

The engine supports different execution modes (e.g., manual, production), execution orders (v0, v1), and both full and partial workflow runs. It is designed to be extensible through a system of lifecycle hooks, allowing for custom logic to be injected at various points in the execution process.

Core Concepts

WorkflowExecute Class

The central class that manages a single workflow run. It is instantiated for each execution and holds the state, data, and logic required to process the workflow graph.

• run(workflow, startNode, destinationNode): The main entry point to start a full workflow execution.
• runPartialWorkflow2(workflow, runData, pinData, dirtyNodeNames, destinationNode): Entry point for partial executions, used in the editor to re-run only a subset of nodes.
• processRunExecutionData(workflow): The core loop that iterates through the nodeExecutionStack and executes each node.
• processSuccessExecution(...): Handles the final state of a successful execution, including saving data and running post-execution hooks.

Sources: packages/core/src/execution-engine/workflow-execute.ts

Execution Data Structures

The engine uses several key data structures to manage state and data flow.

Structure	Description	Source File
IRun	Represents the entire execution run, containing status (success, error, waiting), start/stop times, mode, and the IRunExecutionData.	packages/core/src/execution-engine/tests/workflow-execute.test.ts:31-43
IRunExecutionData	Contains all data related to the execution, including resultData (output of each node) and executionData (context, stack, etc.).	packages/core/src/execution-engine/tests/workflow-execute.test.ts:28-43
IRunData	A map where keys are node names and values are arrays of ITaskData, representing the output of each node run.	packages/core/src/execution-engine/tests/workflow-execute.test.ts:33
IExecuteData	An object passed to each node's execute method, containing the node itself, its input data, and source information.	packages/core/src/execution-engine/workflow-execute.ts:1821-1826

Execution Order

The engine supports two main execution orders, which can be configured in the workflow settings.

1. v0 (Legacy): A depth-first execution strategy.
2. v1 (Default): A breadth-first execution strategy that processes nodes level by level.

The execution order is determined by the executionOrder property in the workflow settings.

Sources: packages/core/src/execution-engine/tests/workflow-execute.test.ts:89, packages/core/src/execution-engine/tests/workflow-execute.test.ts:175

Full Execution Flow

A full workflow execution is initiated by the run method. The engine first checks if the workflow is ready for execution, then processes the nodes in the determined order.

The diagram below illustrates the high-level flow of a full workflow execution.

1. checkReadyForExecution

Before execution, this method validates the workflow to ensure all nodes are known and have their required parameters met. It returns null if the workflow is ready, or an object detailing the issues if not.

Sources: packages/core/src/execution-engine/workflow-execute.ts:1672-1711

2. processRunExecutionData

This is the main execution loop. It processes the nodeExecutionStack until it is empty. For each node in the stack, it:

1. Ensures the node has input data.
2. Runs the nodeExecuteBefore lifecycle hook.
3. Executes the node via runNode.
4. Handles the node's output, including routing for error outputs.
5. Runs the nodeExecuteAfter lifecycle hook.

Sources: packages/core/src/execution.engine/workflow-execute.ts:333-494

3. runNode

This method is responsible for executing a single node. It determines the node type and calls the appropriate method (execute, trigger, poll, etc.). It also handles custom operations defined on nodes.

For trigger nodes, it may return a ITriggerResponse which includes functions to close the trigger or manually trigger it.

Sources: packages/core/src/execution-engine/workflow-execute.ts:502-632

Partial Execution

Partial execution is a key feature for the UI, allowing users to re-run a workflow from a specific point. The runPartialWorkflow2 method orchestrates this.

The logic involves:

1. Creating a subgraph of the workflow containing only the nodes that need to be executed (from the start/dirty nodes up to the destination node).
2. Cleaning the existing runData to remove data for dirty nodes and their descendants.
3. Recreating the nodeExecutionStack for the subgraph.
4. Calling processRunExecutionData to execute the partial workflow.

The following diagram shows the logic for partial execution.

Lifecycle Hooks

The execution engine is instrumented with lifecycle hooks that allow for observing and interacting with the execution process. These are managed by the ExecutionLifecycleHooks class.

The sequence diagram below shows when the primary hooks are triggered during a simple, two-node workflow execution.

Available Hooks

Hook Name	Triggered
workflowExecuteBefore	Before the entire workflow execution begins.
workflowExecuteAfter	After the entire workflow execution has finished (successfully or not).
nodeExecuteBefore	Before a single node is executed.
nodeExecuteAfter	After a single node has been executed.
sendChunk	When a chunk of data (e.g., an error message) should be sent.

Sources: packages/core/src/execution-engine/execution-lifecycle-hooks.ts:12-20

Special Operations & State Management

Waiting Executions

Some nodes, like the GitHub node with its dispatchAndWait operation, can pause a workflow execution and wait for an external event. This is managed through the putExecutionToWait function.

1. The node calls this.putExecutionToWait(WAIT_INDEFINITELY).
2. The engine generates a resumeUrl and passes it to the node's context.
3. The node provides this URL to the external service (e.g., as an input to a GitHub Action).
4. The workflow execution status is set to waiting.
5. When the external service calls the resumeUrl, the workflow resumes from the waiting node.

// Example from Github.node.ts
// ...
const resumeUrl = this.getWorkflowDataProxy(0).$execution.resumeUrl;

body.inputs = {
    ...inputs,
    resumeUrl,
};

// ...
responseData = await githubApiRequest.call(this, 'POST', endpoint, body);
// ...

await this.putExecutionToWait(WAIT_INDEFINITELY);
return [this.getInputData()];

Sources: packages/nodes-base/nodes/Github/Github.node.ts:1270-1298

Error Handling

The engine has robust error handling:

• A top-level try...catch block in processRunExecutionData catches errors during node execution.
• The processSuccessExecution method can receive an error object, which it then attaches to the final IRun data.
• The handleNodeErrorOutput method specifically handles nodes that have a dedicated error output, routing items containing an error property to that output.

// from workflow-execute.ts
// Simplified logic for handleNodeErrorOutput
for (const item of nodeSuccessData[0]) {
    if (item.json.error !== undefined && item.json.message !== undefined) {
        // This is an error item
        errorData.push(item);
    } else {
        // This is a success item
        successData.push(item);
    }
}
nodeSuccessData[0] = successData;
nodeSuccessData[node.description.outputs.length - 1] = errorData;

Sources: packages/core/src/execution-engine/workflow-execute.ts:634-704, packages/core/src/execution-engine/workflow-execute.ts:1777-1819

CLI Integration

The workflow execution engine is utilized by the CLI to run workflows from the command line, particularly for testing.

• packages/cli/src/workflow-runner.ts: This file contains the logic for the n8n execute command. It loads a workflow, creates a WorkflowExecute instance, and runs it.
• packages/testing/playwright/tests/cli-workflows/workflow-tests.spec.ts: This test suite uses execSync to call the n8n execute command and validates the output, including schema validation against snapshots. This provides an end-to-end test of the execution engine from a user's perspective.

This integration demonstrates how the WorkflowExecute class serves as the universal entry point for running workflows, whether initiated from the UI, a webhook, or the command line.

Sources: packages/cli/src/workflow-runner.ts, packages/testing/playwright/tests/cli-workflows/workflow-tests.spec.ts

Conclusion

The Workflow Execution Engine is a sophisticated system for orchestrating the execution of complex, graph-based workflows. Through its WorkflowExecute class, it manages data flow, state, and errors for both full and partial runs. Its support for different execution orders, lifecycle hooks, and special states like "waiting" makes it a flexible and powerful core component of the n8n platform. The engine's design, which is clearly visible in its implementation and extensive test suite, prioritizes correctness, state management, and observability.