Pipeline Orchestrator

Relevant source files

The following files were used as context for generating this wiki page:

• src/engine/orchestrator.ts
• src/engine/index.ts
• src/engine/pipeline.ts
• src/engine/optimizationLoop.ts
• README.md
• plan.md

The Pipeline Orchestrator is the central engine of the SynthCrowd platform, responsible for managing the end-to-end lifecycle of a synthetic user testing run. It coordinates several specialized modules—including the ICP Interpreter, Population Generator, Prompt Compiler, and Agent Runner—to transform high-level natural language descriptions of a customer profile into executable browser agent simulations.

The primary entry point for this system is the runFullPipeline function, which handles the sequential execution of stages while managing external integrations such as real-time event streaming to Convex and performance tracing via Laminar. Beyond standard execution, the orchestrator supports iterative optimization loops where failure analysis leads to automated code modifications and re-testing.

Sources: src/engine/orchestrator.ts:36-45, README.md:14-25, plan.md:14-43

Core Pipeline Architecture

The orchestration logic follows a structured 5-stage pipeline. It begins with data ingestion and ends with the synthesis of a comprehensive friction report.

Execution Stages

1. Interpretation: Converts raw user input (NL or URL) into a structured PersonaConfig representing the statistical center of the Ideal Customer Profile (ICP).
2. Population Generation: Creates a diverse set of N personas based on the center point and a defined variance.
3. Prompt Compilation: Transforms static persona configurations into rich system prompts containing specific behavioral rules.
4. Agent Execution: Launches autonomous browser agents (via Browser Use SDK) in parallel batches to navigate the target site.
5. Reporting: Aggregates step-by-step event data into a final report identifying conversion rates and UX friction points.

Sources: src/engine/orchestrator.ts:54-129, README.md:27-101, plan.md:14-43

Workflow Diagram

The following diagram illustrates the standard data flow through the orchestrator's stages.

Technical Interfaces

The orchestrator utilizes strict TypeScript interfaces to define the inputs required for a test run and the resulting data structure.

Pipeline Configuration

Parameter	Type	Description
user_input	string	NL description or URL for ICP interpretation.
target_url	string	The website to be tested.
goal	string	The conversion target (e.g., "Sign up").
population_size	number	Number of agents to simulate.
variance	number	Statistical spread of persona traits (0-1).
batch_size	number	Number of agents to run concurrently (default: 5).

Sources: src/engine/orchestrator.ts:18-27, src/engine/pipeline.ts:7-13

Data Output Structure

The orchestrator returns a FullPipelineOutput object containing the following:

• run_id: Unique identifier for the current execution.
• agent_results: An array of AgentRunResult detailing the journey of every persona.
• report: A ReportOutput containing synthesized friction points and recommendations.
• compiled_agents: The final prompts used to drive the LLM agents.

Sources: src/engine/orchestrator.ts:29-34, src/engine/index.ts:16-25

Optimization Loop

The orchestrator can be extended into an OptimizationLoop, which introduces a feedback mechanism for automated site improvement. When conversion rates fall below a specified threshold, the system triggers Phase 4 and 5 of the build plan.

Key Optimization Components

• Code Analysis: Inputting failure reasoning and site code to propose minimal, targeted fixes.
• Safe Modification: Utilizing Git-style commits and reverts to ensure the site remains functional after changes.
• Persistence: Storing personas in MongoDB via session_id to ensure consistent testing across iterative reruns.

Sources: src/engine/optimizationLoop.ts:257-300, plan.md:45-70, src/engine/index.ts:11-23

Integration and Tracing

The Orchestrator provides native support for observability and real-time data persistence through a provider-agnostic configuration block.

// src/engine/orchestrator.ts:36-41
export async function runFullPipeline(
  input: FullPipelineInput,
  config: {
    convex_url?: string;
    laminar_api_key?: string;
    browser_use_api_key?: string;
  } = {}
)

If a laminar_api_key is provided, the orchestrator wraps internal calls with tracedInterpretICP, tracedCompilePersona, and tracedReportGeneration. This allows developers to monitor LLM latency and token costs across the entire execution chain.

Sources: src/engine/orchestrator.ts:46-77, plan.md:109-112

Summary

The Pipeline Orchestrator is the foundational component that enables SynthCrowd to move beyond simple browser automation into a comprehensive UX testing and optimization platform. By managing the flow from interpretation to reporting, and potentially looping back through code modification, it provides a closed-loop system for improving conversion rates based on synthetic user feedback.

Sources: src/engine/orchestrator.ts:153-161, README.md:6-12, plan.md:6-12