High-Level Architecture

Relevant source files

The following files were used as context for generating this wiki page: - [README.md](https://github.com/eenlars/lucky/blob/HEAD/README.md) - [packages/core/README.md](https://github.com/eenlars/lucky/blob/HEAD/packages/core/README.md) - [packages/shared/README.md](https://github.com/eenlars/lucky/blob/HEAD/packages/shared/README.md) - [packages/tools/README.md](https://github.com/eenlars/lucky/blob/HEAD/packages/tools/README.md) - [packages/core/src/utils/persistence/README.md](https://github.com/eenlars/lucky/blob/HEAD/packages/core/src/utils/persistence/README.md) - [apps/web/src/lib/api/README.md](https://github.com/eenlars/lucky/blob/HEAD/apps/web/src/lib/api/README.md) - [apps/web/src/lib/workflow_setup_examples/my-workflow-judge.json](https://github.com/eenlars/lucky/blob/HEAD/apps/web/src/lib/workflow_setup_examples/my-workflow-judge.json) - [packages/adapter-supabase/REFACTORING.md](https://github.com/eenlars/lucky/blob/HEAD/packages/adapter-supabase/REFACTORING.md)

The Lucky framework is an autonomous, multi-agent system designed for the evolutionary optimization of complex workflows. It treats workflows as evolvable data structures, specifically directed acyclic graphs (DAGs), where nodes represent AI agents. These agents collaborate by passing structured messages, using a unified tool system, and sharing memory to achieve a common goal. The system's core purpose is to discover effective agent collaboration patterns automatically, reducing the need for manual workflow design.

The architecture is built on principles of self-composition, reliability, and developer experience. Workflows are defined in JSON, allowing them to be programmatically manipulated by genetic algorithms and iterative improvement loops. The entire system is a TypeScript monorepo, emphasizing type-safety and modularity.

Sources: README.md, packages/core/README.md

Monorepo Structure

The project is organized as a monorepo with a clear separation between applications, core logic, and shared packages. This structure facilitates code reuse and independent development of different system parts.

Sources: README.md:19-31

This diagram shows the dependency flow, with apps depending on packages, and most packages depending on the foundational @lucky/shared package.

Sources: README.md:19-31

Path	Description
apps/web	The Next.js user interface for visualizing and managing workflows.
apps/examples	Contains example tool and workflow definitions.
packages/shared	The foundational package containing Zod contracts, types, and utilities. All other packages depend on this.
packages/models	A multi-provider model registry for services like OpenAI and Anthropic.
packages/tools	The unified tool framework, supporting both TypeScript functions and external MCP tools.
packages/core	The workflow execution engine, responsible for DAG orchestration, agent management, and evolution.
packages/adapter-supabase	The persistence layer implementation for Supabase, handling all database interactions.
tests/e2e-essential	Contains smoke and gate tests for end-to-end workflow validation.

Sources: README.md:19-31, packages/shared/README.md, tests/README.md

Core Architectural Layers

The system is designed with five core architectural layers that build upon each other to create a self-composing workflow platform.

Sources: packages/core/README.md:20-43

This diagram illustrates the five core architectural layers of the Lucky platform.

Sources: packages/core/README.md:20-43

1. Agent Network Layer: The base layer consisting of autonomous AI agents (nodes) that communicate via type-safe messages, use tools, and access shared memory.
2. Execution Engine: Orchestrates the workflow as a DAG, managing a three-phase execution pipeline (Prepare → Execute → Process) with error recovery.
3. Evolution System: Enables autonomous improvement through two methods: Cultural Learning (iterative analysis) and Genetic Programming (population-based optimization).
4. Memory & Persistence: Handles large-scale data with chunked storage, AI-powered summarization, and support for multiple backends (in-memory, filesystem, cloud).
5. Validation & Quality: A multi-layer validation system that uses Zod schemas for type safety and AI-powered checks for real-time quality assessment and automatic repair.

Sources: packages/core/README.md:20-43

Module Breakdown

@lucky/shared (Contracts & Types)

This is the foundational package with no internal dependencies. It provides the single source of truth for all data structures using Zod schemas. This approach ensures runtime validation and compile-time type safety across the entire monorepo.

• Contracts (/contracts): Zod schemas for all core concepts like Workflow, Agent, Tool, and Message.
• Types (/types): Auto-generated TypeScript types inferred from Zod schemas and the Supabase database schema.
• Utilities: Shared helper functions like type guards (isNir) and ID generators (genShortId).

Sources: packages/shared/README.md

@lucky/tools (Unified Tool Ecosystem)

The tool system is a hybrid framework that allows agents to interact with both internal code and external processes.

• Code Tools: TypeScript functions that are auto-discovered from the filesystem. They are defined using a defineTool() factory, which provides type-safety, context access, and automatic error handling.
• MCP (Model Context Protocol) Tools: External servers that communicate via the MCP standard. This allows for language-agnostic tools (e.g., Python scripts) to be integrated into workflows.
• Intelligent Selection: An AI-driven strategy (selectToolStrategy.ts) analyzes the agent's prompt and context to decide which tools to use (auto, required, none, or specific tools).

Sources: packages/tools/README.md, README.md:120-135, [packages/core/README.md:209-231]

@lucky/core (Execution & Evolution Engine)

This is the heart of the system, orchestrating agent workflows and driving their evolution.

Workflow Engine (/workflow)

The engine executes workflows defined as DAGs. It manages the graph structure, validates connectivity, detects cycles, and orchestrates the flow of messages between agent nodes.
Sources: [packages/core/README.md:288-316]

Node System (/node)

Each node is an autonomous agent with its own configuration, memory, and tool access. Node invocation follows a three-phase pipeline:

1. Prepare: Sets up context and selects a tool strategy.
2. Execute: Interacts with an AI model and coordinates tool usage.
3. Process: Validates the response and makes a handoff decision to the next node.
   Sources: [packages/core/README.md:188-208]

Evolution System (/improvement)

This module enables workflows to self-improve.

• Cultural Evolution: An iterative process where a single workflow is analyzed for performance, and targeted improvements are applied and validated.
• Genetic Programming: A population-based approach where workflows (genomes) are evolved over generations using crossover and mutation operators. Fitness is evaluated based on correctness, cost, and speed.
  Sources: packages/core/README.md:49-103, [README.md:167-183]

Persistence (@lucky/adapter-supabase & /utils/persistence)

The persistence layer is designed for comprehensive tracking of workflow execution and evolution, with a multi-backend architecture. The primary implementation uses Supabase.

• Genealogy Tracking: Every workflow version is stored immutably, tracking its evolutionary parents (crossover, mutation) and context (generation, run ID).
• Context Storage: A ContextStore interface provides a key-value store for workflow and node-scoped data. Implementations include an in-memory store and a Supabase store that handles automatic chunking and summarization for large data.
• Database Schema: The schema is designed to capture the entire lifecycle, from workflow definition to individual node invocations, including costs and status.

Sources: packages/core/src/utils/persistence/README.md, packages/adapter-supabase/REFACTORING.md

The following diagram shows the Entity-Relationship model for the core persistence tables.

apps/web (User Interface & API)

The web app is a Next.js application that serves as the UI for the system. It includes a type-safe API layer for communication with the backend.

• API Layer (/lib/api): A pattern that uses Zod schemas from @lucky/shared as the single source of truth for API contracts. This provides end-to-end type safety, runtime validation, and removes boilerplate code. The handleBody() and alrighty() helpers ensure that server routes only process and return data that conforms to the predefined schemas.
• Visualization: The UI provides real-time visualization of workflow execution graphs and evolutionary fitness metrics.

Sources: apps/web/src/lib/api/README.md, README.md:20-22

Execution Flow

A typical workflow execution follows a series of steps, from initialization to evolution, orchestrated by the core engine.

This flowchart shows the lifecycle of a workflow run.

1. Workflow Initialization: A workflow is created from a JSON configuration.
2. Node Setup & Tool Discovery: Agents (nodes) are initialized, and their required tools are discovered and registered.
3. Message Queue Processing: The execution starts, processing messages from a queue.
4. Node Execution: An agent receives a message and executes its three-phase pipeline.
5. Memory Management: During execution, agents can read from and write to shared memory. Large data is automatically chunked and summarized.
6. Handoff & Routing: After execution, the agent's AI makes a decision on which node to hand off to next.
7. Response Generation: The final result is aggregated, validated, and returned.
8. Evolution & Improvement: After the run, performance is analyzed, and the workflow configuration is updated via cultural or genetic optimization.

*Sources: packages/core/README.md:364-385