Testing Strategy & Automation Tests

Relevant source files

The following files were used as context for generating this wiki page:

• tests/convex/setup.ts
• python/tests/test_workflow_regressions.py
• vitest.convex.config.ts
• tests/convex/workflows.test.ts
• tests/convex/crons.test.ts
• tests/convex/helpers.ts
• tests/convex/http.test.ts
• frontend/scripts/workflow-regression-proof.ts
• frontend/scripts/workflow-import-e2e.ts
• AGENTS.md

Testing in the bot-auto-ig project is structured around a multi-layered approach that ensures the integrity of the backend (Convex), the automation runtime (Python), and the frontend integration. The strategy combines unit tests, integration tests for HTTP endpoints, cron job verification, and regression scripts to safeguard core workflows and data handling.

The project mandates adding or updating tests when changing automation behavior, parsing, retries, or state handling. Specific rules include using 2-space indentation for TypeScript tests and following PEP 8 for Python tests.
Sources: AGENTS.md:65-72

Backend Testing (Convex)

The Convex backend is tested using convex-test and vitest. These tests verify mutations, queries, and complex logic such as workflow transitions and artifact cascading.

Test Environment & Helpers

A centralized testing harness is provided via createConvexTest, which initializes a test environment with a predefined UserIdentity. This identity simulates a logged-in user with Clerk-based token identifiers.
Sources: tests/convex/helpers.ts:13-26

The testing suite uses several helper functions to seed data:

• seedList: Creates a list of leads.
• seedProfile: Creates an Instagram profile with optional overrides for proxies and cookies.
• seedWorkflow: Initializes a workflow with default idle status and daily schedules.
  Sources: tests/convex/helpers.ts:39-86

Workflow Logic & State Transitions

Tests verify that workflows correctly transition through statuses: pending -> running -> paused -> completed. They also ensure that deleting a workflow triggers a cascade delete for associated artifacts and storage objects.
Sources: tests/convex/workflows.test.ts:12-42, tests/convex/workflows.test.ts:114-142

HTTP API & Security Testing

The project includes an internal HTTP surface used by the Python automation runtime. Testing ensures these routes are protected by an INTERNAL_API_KEY and that data is correctly mapped between Python's snake_case and the backend's expected formats.

Authentication & Authorization

Tests confirm that:

1. Requests are rejected with 401 Unauthorized if the INTERNAL_API_KEY is incorrect.
2. Requests fail with 500 if the key is not configured in the environment.
3. Successful requests return mapped JSON data.
   Sources: tests/convex/http.test.ts:17-49

Data Mapping

The HTTP layer handles the translation of profile payloads. For example, daily_scraping_limit from a POST request is mapped to the internal dailyScrapingLimit field. Tests also verify that sensitive fields like cookies_json are omitted from list views but included in detail responses.
Sources: tests/convex/http.test.ts:51-84, tests/convex/http.test.ts:132-159

Automation & Regression Scripts

Beyond standard unit tests, the project uses specialized scripts to prevent regressions in critical infrastructure and workflow import/export logic.

Workflow Regression Proofing

A dedicated script (workflow-regression-proof.ts) checks for unexpected changes in "Runtime Protocol" files. It prevents modification of files like server/api/workflows.ts and python/getting_started/run_workflow.py without explicit intent. It also asserts that:

• Mutation wiring for create, update, and duplicate remains intact.
• The import feature remains "create-only" (no overwrite path).
• Export actions are disabled when a workflow is running.
  Sources: frontend/scripts/workflow-regression-proof.ts:18-63

Import/Export E2E Scenarios

The workflow-import-e2e.ts script executes various scenarios to validate the robustness of the workflow migration system:

Scenario Type	Validation Detail
Happy Path	Valid JSON import with correct format and version.
Auto-Rename	Workflows with duplicate names are suffixed with a timestamp.
Warnings	Missing list IDs in "Select List" nodes trigger warnings but don't block import.
Hard Failures	Rejection of invalid file extensions (.txt), unknown activity IDs, or missing edge endpoints.
Caps	Enforcement of limits on file size (2MB) and node count (500).
Sources: frontend/scripts/workflow-import-e2e.ts:40-192

Scheduled Tasks (Crons)

Testing for cron jobs ensures that background maintenance tasks are registered and execute the correct internal functions.

Tests verify the logic of these targets, such as ensuring that autoUnsubscribe correctly updates the status of accounts that have been subscribed for longer than 8 days.
Sources: tests/convex/crons.test.ts:71-84

Summary of Testing Commands

The project uses several test runners depending on the module:

Command	Purpose
npm run test:convex	Runs Vitest for all Convex backend modules.
python -m pytest python/tests -q	Executes Python automation tests.
npm --prefix frontend run lint	Ensures code style compliance in the UI.
Sources: AGENTS.md:54-63

The testing strategy is designed to ensure that the complex interplay between the React frontend, Convex backend, and Python automation engine remains stable during rapid development. By combining state machine validation, HTTP security checks, and protocol regression scripts, the system maintains reliability for long-running automation tasks.