Event Parsing & Automation State
Relevant source files
The following files were used as context for generating this wiki page:
The Event Parsing and Automation State system is responsible for managing the lifecycle, persistence, and real-time communication of Instagram automation tasks. It ensures that the server can recover from crashes by persisting state to the filesystem, while simultaneously providing a real-time event stream to the frontend via WebSockets.
This system acts as the bridge between the high-level workflow orchestration and the low-level Python automation processes. It handles the extraction of structured events from process logs, maintains a consistent status (idle, running, stopping), and manages the lifecycle of automation runners.
Automation State Management
The server maintains a persisted state to handle crash recovery and track active automation processes. This state is stored in server/data/automation_state.json and includes process IDs, start times, and execution settings.
Persisted State Structure
The PersistedState interface defines the schema for the automation status on disk.
| Field | Type | Description |
|---|---|---|
status | idle | running | stopping | The current operational mode of the automation service. |
pid | number | null | The Process ID of the active Python automation runner. |
startedAt | string | null | ISO timestamp of when the automation run began. |
settings | Record<string, unknown> | null | The configuration parameters passed to the runner. |
Sources: server/automation/state.ts:24-29, server/automation/state.ts:34-39
Lifecycle Transitions
State is updated through specific utility functions that ensure atomic file writes to prevent corruption during crashes.
markStarted(pid, settings): transitions status torunningand saves the PID.markStopped(): Clears the persisted state file.detectInterruptedRun(): On server startup, checks if a previous session left a process inrunningstate with a valid PID.
Sources: server/automation/state.ts:88-108, server/index.ts:145-163
The diagram above illustrates the recovery logic during server initialization. Sources: server/index.ts:145-163
Event Parsing Architecture
The system parses output from the Python automation runners to generate structured events. These events are used to update the UI and internal state.
Event Extraction Logic
The extractAutomationEvent function handles two primary types of message formats:
- Direct Payloads: JSON objects that already contain event types (e.g.,
profile_started). - Embedded Log Events: Plain text log lines that contain a specific
__EVENT__prefix followed by a JSON string.
Sources: server/automation/event-parser.test.ts:7-30
Real-time Communication Flow
The AutomationService on the server and useWebSocket hook on the frontend collaborate to synchronize state and logs.
This flow shows how automation logs are transformed into UI updates. Sources: server/automation/routes.ts:119-145, frontend/src/hooks/useWebSocket.ts:114-135
Automation Service & Routes
The automation is controlled via REST API endpoints and monitored through an AutomationService class that maintains the socket connection and status.
API Endpoints
| Endpoint | Method | Description |
|---|---|---|
/api/automation/status | GET | Returns the current status and running boolean. |
/api/automation/start | POST | Acquires mutex, spawns Python process, and saves PID. |
/api/automation/stop | POST | Signals the process to stop and updates status to stopping. |
/api/automation/login | POST | Specifically triggers a login session for a profile. |
Sources: server/automation/routes.ts:25-78
AutomationService (Server-side)
The AutomationService class acts as an internal client that connects to the local WebSocket to monitor its own process events.
status: Getter for current service state (idle,starting,running,stopping,error).connect(): Establishes the WebSocket connection to the server's own/wsendpoint.start(settings): Sends the POST request to trigger the automation process.
Sources: server/automation/runner.ts:16-55
Database Schema (Convex)
The state of workflows and their artifacts is persisted in the Convex database. This differs from the "Automation State" (process state) by tracking the logical progress of the automation's business goals.
Workflow Execution Fields
In the workflows table, several fields track the automation's progress through a graph:
status: Current execution status (e.g.,pending,running,completed,failed).currentNodeId: The specific node in the ReactFlow graph currently being executed.nodeStates: A map of individual node IDs to their specific execution metadata (e.g., scrape progress).
Sources: convex/schema.ts:117-130
Workflow Artifacts
Results generated during an automation run are stored as artifacts, linked back to the workflow and node that produced them.
| Field | Type | Description |
|---|---|---|
stats | object | Tracks scraped, deduped, and targetsCompleted. |
storageId | id("_storage") | Link to the raw data file in Convex storage. |
sourceProfileName | string | The Instagram profile that performed the action. |
Sources: convex/schema.ts:153-172
Summary
The Event Parsing & Automation State module provides a resilient framework for executing long-running tasks. By separating Process State (handled via server/automation/state.ts for crash recovery) from Workflow State (handled via convex/schema.ts for logic persistence), the system ensures that automation can be monitored in real-time while remaining robust against infrastructure failures. Real-time feedback is facilitated by a standardized event parsing protocol that bridges Python process outputs to React frontend state.