High-Level Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The bot-auto-ig project is a multi-tier automation platform designed for Instagram data scraping and interaction. Its architecture is divided into a React-based frontend for workflow orchestration, a Convex backend for real-time data management and scheduling, an Express-based server for API routing, and a Python automation runtime that executes browser-level actions.
The system centers around a "Workflow" model, where users define sequences of activities (nodes) and connections (edges) that are executed by automated agents. Data generated by these workflows is stored as artifacts, which can be exported or processed further within the platform.
Sources: AGENTS.md:1-55, convex/schema.ts:55-150
System Component Overview
The project is structured into four primary domains, each responsible for a specific layer of the automation lifecycle.
Domain Responsibilities
| Component | Responsibility | Primary Technologies |
|---|---|---|
| Frontend | Workflow design (React Flow), monitoring, and account management. | React, TypeScript, React Flow |
| Server | API routing, WebSocket communication, and authentication bridging. | Node.js, Express |
| Convex Backend | Persistence, scheduling (Crons), and real-time state synchronization. | Convex, TypeScript |
| Python Runtime | Browser automation, Instagram action execution, and local database syncing. | Python, Playwright/Selenium |
Sources: AGENTS.md:40-55, frontend/src/features/workflows/components/WorkflowFlowEditor.tsx:40-60
Architecture Diagram
The following diagram illustrates the high-level interaction between the user interface, the central data layer, and the automation execution environment.
Data Architecture and Persistence
The system uses a strictly typed schema managed by Convex. Data is organized into tables that track automation assets, including profiles, accounts, and the workflow definitions themselves.
Core Schema Entities
- Workflows: Stores the ReactFlow graph (nodes/edges) and scheduling configurations (interval, daily, cron).
- Workflow Artifacts: Stores results from scraping activities, including metadata and links to Convex
_storage. - Profiles: Represents browser profiles with unique fingerprints, proxy settings, and session cookies.
- Instagram Accounts: Tracks the state and assignment of specific Instagram handles to profiles.
Sources: convex/schema.ts:1-150
Workflow Schema Details
The workflow table is the most complex entity, capturing definition, scheduling, and execution state.
| Field | Type | Description |
|---|---|---|
nodes | v.any() | ReactFlow nodes array (positions, configs). |
edges | v.any() | ReactFlow edges array (connections). |
status | union | Workflow state: idle, pending, running, completed, etc. |
scheduleConfig | object | Config for scheduling (interval, day, minute, cronspec). |
nodeStates | v.any() | Map of nodeId to its specific execution progress. |
Sources: convex/schema.ts:75-120
Execution Model
The system operates on an event-driven and scheduled execution model. Workflows can be triggered manually (via the UI) or automatically (via Convex crons).
Workflow Execution Flow
A typical workflow execution involves several steps from the UI to the automation runner.
Scheduling and Scaling
Workflows support complex scheduling options, including standard intervals and cron expressions. The backend handles "daily resets" for workflows with run limits (maxRunsPerDay).
- Crons: Managed via
@convex-dev/crons, stored incronJobId. - Concurrency: Managed through
parallelProfileswithin the activity configurations. - Lease Management: For scraping tasks, a lease system (
profilesClaimScrapeLease) ensures that only one worker operates on a specific profile at a time to prevent account flags.
Sources: convex/schema.ts:85-110, server/shared/convexClient.ts:250-280
Automation Runtime (Python Domain)
The Python runtime acts as the browser automation engine. It communicates with the backend via the Express API server (authorized by INTERNAL_API_KEY).
- Runners: Entry points located in
python/runners/initiate automation tasks. - Actions: Instagram-specific actions (e.g., following, scraping, messaging) are implemented in
python/actions/. - Browser Control: Lifecycle management (opening/closing browsers, setting fingerprints) is handled by the
python/browser/core infrastructure. - Database Client: A specialized client (
python/database/) synchronizes local state with the central Convex database through HTTP requests.
Sources: AGENTS.md:40-45, server/shared/convexClient.ts:100-150
API and Integration Architecture
The Express server serves as a bridge for non-Convex environments (like Python) to interact with the database. It enforces security through an INTERNAL_API_KEY.
Integration Interface
The following diagram highlights how the server bridges disparate system components.
Critical Integration Endpoints
| Endpoint | Method | Purpose |
|---|---|---|
/api/workflows/run | POST | Triggers a workflow execution in the automation runner. |
/api/workflows/update-status | POST | Updates the current state and progress of a running workflow. |
/api/profiles/sync-status | POST | Updates profile activity status (e.g., using: true). |
/api/workflow-artifacts | GET/POST | Manages output data from scraping nodes. |
Sources: tests/convex/http.test.ts:200-240, server/shared/convexClient.ts:200-350
Conclusion
The high-level architecture of bot-auto-ig is characterized by a decoupled design that separates workflow logic (Frontend), data orchestration (Convex), and browser-level execution (Python). This structure allows for scalable automation where multiple independent runners can execute complex, multi-step workflows while maintaining a centralized, real-time source of truth for configuration and state.