Workflow Runner & Execution Graph
Relevant source files
The following files were used as context for generating this wiki page:
The Workflow Runner and Execution Graph system provides a structured framework for defining, scheduling, and executing automated browser tasks. It utilizes a graph-based approach where users define "workflows" consisting of nodes (representing specific activities like starting a browser or scraping data) and edges (representing the execution flow).
The system is split across three main tiers: a React-based frontend for visual graph editing, a Convex backend for state persistence and scheduling, and an Express-based server that orchestrates the execution of Python-based automation runners.
Workflow Architecture & Data Model
The core of the system is the workflows table, which stores both the structural definition of the graph and its runtime execution state.
Database Schema
The schema defines how graph structures (nodes and edges) are stored alongside scheduling configurations and execution telemetry.
| Field | Type | Description |
|---|---|---|
nodes | v.any() | Array of ReactFlow nodes containing positions and activity configurations. |
edges | v.any() | Array of ReactFlow edges defining the sequence of activities. |
status | v.union(...) | Current state: idle, pending, running, paused, completed, failed, or cancelled. |
nodeStates | v.any() | A map of nodeId to its specific execution progress or result. |
scheduleType | v.union(...) | Trigger type: interval, daily, weekly, monthly, cron, or instant. |
Sources: convex/schema.ts:80-137
Execution State Machine
Workflows transition through specific statuses during their lifecycle. Transitions are strictly validated to ensure data integrity.
The state machine ensures that a workflow cannot be updated or deleted while in a running state.
Sources: convex/workflows/mutations.ts:47-50, convex/workflows/mutations.ts:167-185
Scheduling and Triggering
The scheduling system uses the @convex-dev/crons component to register recurring jobs based on user configuration.
Scheduling Flow
When a workflow is activated, the system calculates the next run time or registers a cron job. If the scheduleType is instant, it triggers an immediate execution.
Server-Side Execution Orchestration
The Express server (server/workflows/service.ts) acts as the bridge between the backend state and the OS-level Python processes.
Runner Process Management
When the /api/workflows/run endpoint is hit, the server performs the following:
- Mutex Acquisition: Uses an
automationMutexto prevent race conditions and enforce concurrency limits (default max 3). - Payload Construction: Builds a JSON payload containing the full graph definition and current
nodeStates. - Process Spawning: Executes the Python runner script (
python/runners/run_workflow.py) as a subprocess. - Event Wiring: Streams
stdoutandstderrfrom the process, parsing them into structured logs and status updates.
Sources: server/workflows/routes.ts:107-122, server/workflows/service.ts:251-285
Real-time Monitoring
The server monitors the Python subprocess and broadcasts updates via WebSockets to the frontend.
Frontend Flow Editor
The editor provides a visual interface for constructing the execution graph using reactflow.
Graph Composition Logic
The editor manages nodes and edges through a custom hook, useFlowEditorState.ts, which handles:
- Insertion Context: Logic for adding nodes either as "disconnected" blocks or automatically connected to a source node.
- Activity Configuration: Mapping UI settings to the
data.configobject within each node. - Viewport Persistence: Saving the user's zoom level and position in
localStoragekeyed byworkflowId.
| Component | Responsibility |
|---|---|
WorkflowFlowEditor | Main wrapper and provider for the ReactFlow canvas. |
EditorCanvas | Renders the nodes and handles connection/drag events. |
NodeSettingsPanel | Side panel for editing specific activity parameters (e.g., parallel_profiles). |
BlockLibraryDialog | Searching and selecting activities to add to the graph. |
Sources: frontend/src/features/workflows/components/WorkflowFlowEditor.tsx:55-104, frontend/src/features/workflows/hooks/useFlowEditorState.ts:63-95
Node Connection Example
When a user connects two nodes, the system automatically assigns default edge styles and markers.
const onConnect = useCallback(
(connection: Connection) => {
setEdges((prev) => addEdge({ ...WORKFLOW_EDGE_DEFAULTS, ...connection }, prev))
},
[setEdges],
)
Sources: frontend/src/features/workflows/hooks/useFlowEditorState.ts:272-277
Summary
The Workflow Runner & Execution Graph system effectively decouples high-level task definition from low-level automation execution. By using a persistent graph structure in Convex, the system allows for complex multi-step automations that can be visually managed by users while maintaining robust process management and real-time feedback on the server.