/eenlars/lucky

Wiki directory

Browse AI-generated documentation for any repository.

Developer Guide

Relevant source files

The following files were used as context for generating this wiki page:

This guide provides a comprehensive overview of the lucky project, covering its architecture, setup, development practices, and core components. The system is a framework for self-improving agentic workflows, which uses evolutionary optimization to discover effective agent collaboration patterns. Workflows are defined as evolvable JSON data structures, optimized through genetic programming and iterative improvement.

The core of the project is an autonomous workflow system that operates as a directed acyclic graph (DAG) of AI agents. These agents pass structured messages, use tools through unified interfaces, and evolve their own structure and behavior.

Sources: README.md:5-10, packages/core/README.md:3-6

Project Structure

The project is a monorepo managed with bun. It is organized into apps for runnable applications and packages for shared libraries.

.
├── apps/
│   ├── web/              Next.js UI (port 3000+) - workflows, settings, connectors
│   └── examples/         Example tools & workflow definitions
│
├── packages/
│   ├── shared/          Zod contracts, types, utilities (BUILD FIRST)
│   ├── models/          Multi-provider model registry (OpenAI, Anthropic, etc)
│   ├── tools/           Tool framework (code tools + MCP integration)
│   ├── core/            Workflow execution engine (DAG, nodes, evolution)
│   └── adapter-supabase/ Persistence layer (optional, can use mock)
│
└── tests/
    └── e2e-essential/   Smoke tests (pre-commit) & gate tests (pre-push)

Source: README.md:18-32

The following diagram illustrates the build dependency order of the packages.

Generating architecture diagram...

This diagram shows the build order, which is shared → models / tools / adapter-supabase → core → web / examples.

Sources: README.md:34, packages/shared/README.md:104-109

Installation and Setup

Prerequisites

Bun 1.2+
Node.js 18+
Git

Source: README.md:38

Initial Setup

Clone the repository:

git clone https://github.com/eenlars/lucky.git
cd lucky

Install dependencies: This command also builds the required packages.
```
bun install
```
Configure environment: Copy the example environment file and add your API keys.
```
cp .env.example .env.local
```
You need to fill in your API keys (e.g., OPENROUTER_API_KEY or OPENAI_API_KEY) in .env.local.
Start the web interface:
```
cd apps/web && bun run dev
```

Source: README.md:38-51

Configuration

The system uses a dual environment configuration:

Local Development (SDK/CLI):
- Environment variables are loaded from .env and .env.local.
- MCP (Model Context Protocol) server configurations are read from a mcp-secret.json file, with the path specified by the MCP_SECRET_PATH environment variable.
UI-Based Workflows (Production):
- API keys are encrypted and stored in the Supabase lockbox (lockbox.user_secrets).
- MCP server configurations are stored in the browser's localStorage but are not yet fully integrated with the core execution engine.

Source: README.md:61-76

Common Tasks

The following scripts are available at the root of the monorepo:

Command	Description
`bun run build`	Build all packages and apps.
`bun run tsc`	Run type-checking across the entire project.
`bun run format`	Format all files using Biome.
`bun run format:check`	Check for formatting issues without applying changes.
`bun run test:smoke`	Run fast smoke tests, also used in pre-commit hooks.
`bun run test:gate`	Run comprehensive gate tests, also used in pre-push hooks.

Sources: README.md:53-59, README.md:86-89, package.json:11-20

To run a single workflow execution via the CLI, use the once script in the core package. You can use mock persistence to run without a Supabase instance.

USE_MOCK_PERSISTENCE=true bun -C packages/core run once

Source: README.md:175-176

Tool Architecture

The framework supports two types of tools: Code Tools and MCP Tools.

Code Tools: These are TypeScript functions located in packages/tools/src/definitions/. They are explicitly registered and offer type-safety and direct execution.
MCP Tools: These are external servers that conform to the Model Context Protocol. They are configured via mcp-secret.json and allow for language-agnostic tool integration.

Source: README.md:93-107

The `defineTool` Pattern

All tools are created using the defineTool factory function, which ensures a consistent structure and provides runtime validation, type safety, and automatic error handling.

// From: apps/examples/definitions/todo-manager/tool-todo-write.ts
import { defineTool } from "@lucky/tools"
import { z } from "zod"

const todoWrite = defineTool({
  name: "todoWrite",
  description: "Create and manage structured task lists...",
  params: z.object({
    todos: z.array(todoItemSchema).min(0, "Todos array is required"),
  }),
  async execute(params, externalContext) {
    // ... implementation
  },
})

Source: apps/examples/definitions/todo-manager/tool-todo-write.ts:114-124

Tool Auto-Discovery

Code tools are automatically discovered from the filesystem. The AutoDiscovery module scans for files matching /tool*.ts within the /src/runtime/code_tools/ directory, dynamically imports them, and registers them in the CodeToolRegistry.

Source: packages/core/src/tools/README.md:46-57

Tool Registration Flow

The diagram below shows how a new tool is added to the system.

Generating architecture diagram...

This flow ensures that tool definitions are centralized and type-safe across the application.

Sources: README.md:109-115, packages/shared/README.md:38-46, packages/core/src/tools/README.md:46-57

Testing

The project has a comprehensive testing strategy, including unit, integration, and end-to-end (E2E) tests.

Test Directory Structure

Tests are organized by type in the root tests/ directory.

integration/: For tests involving multiple modules and external dependencies like APIs and databases.
e2e-essential/: For complete workflow tests.
- smoke/: Critical path tests run on every commit.
- gate/: Comprehensive tests run before every push.

Unit tests are co-located with the source code in the respective packages directories.

Source: tests/README.md:5-16

Shared Test Infrastructure

A shared test infrastructure is available in apps/web/src/__tests__/shared/ to reduce boilerplate and improve maintainability. It provides mock functions for various parts of the system.

Key mock setup functions include:

setupGPTestMocks(): Complete setup for genetic programming tests.
setupDatabaseTestMocks(): Mocks the Supabase client for persistence tests.
mockRuntimeConstantsForGP(): Mocks common runtime configurations for GP tests.

Source: apps/web/src/tests/shared/README.md:7-21, 60-80

Writing Tests: Quick Start

Follow this quick start guide from packages/core/src/__tests__/TESTING.md to write new tests.

Copy the starter code:

import { describe, it, expect, beforeEach, vi } from "vitest"
import {
  setupCoreTest,
  mockRuntimeConstantsForGP,
} from "@core/utils/__tests__/setup/coreMocks"

describe("MyFunction", () => {
  beforeEach(() => {
    setupCoreTest() // Always include this
    // Add mockRuntimeConstantsForGP() if your function uses CONFIG.*
  })

  it("should do something", async () => {
    // Your test here
  })
})

Source: packages/core/src/tests/TESTING.md:3-19

Identify dependencies and mock them accordingly (e.g., external APIs, database, file system).
Use the appropriate template for testing evaluators, workflow functions, database operations, or AI API calls.

Running Tests

Command	Description
`bun run test`	Run all unit tests.
`bun run test:smoke`	Run fast smoke tests.
`bun run test:gate`	Run comprehensive gate tests.
`bun run test:integration`	Run only integration tests.
`bunx vitest run <path_to_test>`	Run a specific test file.

Sources: README.md:57-58, tests/README.md:21-29

Persistence Layer

The persistence layer is designed with clean architecture principles, supporting multiple backends like Supabase and an in-memory store for testing.

Factory Pattern

To instantiate the persistence layer, use the createPersistence factory. This allows for dependency injection and easy swapping of backends.

// For production with Supabase
import { createPersistence } from '@together/adapter-supabase'
const persistence = createPersistence({ backend: 'supabase' })

// For testing with in-memory storage
const persistence = createPersistence({ backend: 'memory' })

This pattern is also used for running the application without a database connection by setting the USE_MOCK_PERSISTENCE=true environment variable.

Sources: packages/adapter-supabase/REFACTORING.md:10-18, README.md:175-176

Database Schema

The core database schema tracks workflows, their versions, invocations, and the evolutionary history.

The diagram below outlines the main tables and their relationships.

Generating architecture diagram...

Source: packages/core/src/utils/persistence/README.md:25-72

Code Style

The project uses Biome for code formatting and linting. Configuration is located in biome.json.

To format all files: bun run format
To check for formatting issues: bun run format:check

Path aliases like @core, @shared, and @lucky/tools should be used instead of relative paths to improve readability and maintainability.

Sources: README.md:85-89, biome.json

Developer Guide

Relevant source files

The following files were used as context for generating this wiki page:

Sources: README.md:5-10, packages/core/README.md:3-6

Project Structure

The project is a monorepo managed with bun. It is organized into apps for runnable applications and packages for shared libraries.

.
├── apps/
│   ├── web/              Next.js UI (port 3000+) - workflows, settings, connectors
│   └── examples/         Example tools & workflow definitions
│
├── packages/
│   ├── shared/          Zod contracts, types, utilities (BUILD FIRST)
│   ├── models/          Multi-provider model registry (OpenAI, Anthropic, etc)
│   ├── tools/           Tool framework (code tools + MCP integration)
│   ├── core/            Workflow execution engine (DAG, nodes, evolution)
│   └── adapter-supabase/ Persistence layer (optional, can use mock)
│
└── tests/
    └── e2e-essential/   Smoke tests (pre-commit) & gate tests (pre-push)

Source: README.md:18-32

The following diagram illustrates the build dependency order of the packages.

Generating architecture diagram...

This diagram shows the build order, which is shared → models / tools / adapter-supabase → core → web / examples.

Sources: README.md:34, packages/shared/README.md:104-109

Installation and Setup

Prerequisites

Bun 1.2+
Node.js 18+
Git

Source: README.md:38

Initial Setup

Clone the repository:

git clone https://github.com/eenlars/lucky.git
cd lucky

Install dependencies: This command also builds the required packages.
```
bun install
```
Configure environment: Copy the example environment file and add your API keys.
```
cp .env.example .env.local
```
You need to fill in your API keys (e.g., OPENROUTER_API_KEY or OPENAI_API_KEY) in .env.local.
Start the web interface:
```
cd apps/web && bun run dev
```

Source: README.md:38-51

Configuration

The system uses a dual environment configuration:

Local Development (SDK/CLI):
- Environment variables are loaded from .env and .env.local.
- MCP (Model Context Protocol) server configurations are read from a mcp-secret.json file, with the path specified by the MCP_SECRET_PATH environment variable.
UI-Based Workflows (Production):
- API keys are encrypted and stored in the Supabase lockbox (lockbox.user_secrets).
- MCP server configurations are stored in the browser's localStorage but are not yet fully integrated with the core execution engine.

Source: README.md:61-76

Common Tasks

The following scripts are available at the root of the monorepo:

Command	Description
`bun run build`	Build all packages and apps.
`bun run tsc`	Run type-checking across the entire project.
`bun run format`	Format all files using Biome.
`bun run format:check`	Check for formatting issues without applying changes.
`bun run test:smoke`	Run fast smoke tests, also used in pre-commit hooks.
`bun run test:gate`	Run comprehensive gate tests, also used in pre-push hooks.

Sources: README.md:53-59, README.md:86-89, package.json:11-20

To run a single workflow execution via the CLI, use the once script in the core package. You can use mock persistence to run without a Supabase instance.

USE_MOCK_PERSISTENCE=true bun -C packages/core run once

Source: README.md:175-176

Tool Architecture

The framework supports two types of tools: Code Tools and MCP Tools.

Code Tools: These are TypeScript functions located in packages/tools/src/definitions/. They are explicitly registered and offer type-safety and direct execution.
MCP Tools: These are external servers that conform to the Model Context Protocol. They are configured via mcp-secret.json and allow for language-agnostic tool integration.

Source: README.md:93-107

The `defineTool` Pattern

All tools are created using the defineTool factory function, which ensures a consistent structure and provides runtime validation, type safety, and automatic error handling.

// From: apps/examples/definitions/todo-manager/tool-todo-write.ts
import { defineTool } from "@lucky/tools"
import { z } from "zod"

const todoWrite = defineTool({
  name: "todoWrite",
  description: "Create and manage structured task lists...",
  params: z.object({
    todos: z.array(todoItemSchema).min(0, "Todos array is required"),
  }),
  async execute(params, externalContext) {
    // ... implementation
  },
})

Source: apps/examples/definitions/todo-manager/tool-todo-write.ts:114-124

Tool Auto-Discovery

Source: packages/core/src/tools/README.md:46-57

Tool Registration Flow

The diagram below shows how a new tool is added to the system.

Generating architecture diagram...

This flow ensures that tool definitions are centralized and type-safe across the application.

Sources: README.md:109-115, packages/shared/README.md:38-46, packages/core/src/tools/README.md:46-57

Testing

The project has a comprehensive testing strategy, including unit, integration, and end-to-end (E2E) tests.

Test Directory Structure

Tests are organized by type in the root tests/ directory.

integration/: For tests involving multiple modules and external dependencies like APIs and databases.
e2e-essential/: For complete workflow tests.
- smoke/: Critical path tests run on every commit.
- gate/: Comprehensive tests run before every push.

Unit tests are co-located with the source code in the respective packages directories.

Source: tests/README.md:5-16

Shared Test Infrastructure

A shared test infrastructure is available in apps/web/src/__tests__/shared/ to reduce boilerplate and improve maintainability. It provides mock functions for various parts of the system.

Key mock setup functions include:

setupGPTestMocks(): Complete setup for genetic programming tests.
setupDatabaseTestMocks(): Mocks the Supabase client for persistence tests.
mockRuntimeConstantsForGP(): Mocks common runtime configurations for GP tests.

Source: apps/web/src/tests/shared/README.md:7-21, 60-80

Writing Tests: Quick Start

Follow this quick start guide from packages/core/src/__tests__/TESTING.md to write new tests.

Copy the starter code:

import { describe, it, expect, beforeEach, vi } from "vitest"
import {
  setupCoreTest,
  mockRuntimeConstantsForGP,
} from "@core/utils/__tests__/setup/coreMocks"

describe("MyFunction", () => {
  beforeEach(() => {
    setupCoreTest() // Always include this
    // Add mockRuntimeConstantsForGP() if your function uses CONFIG.*
  })

  it("should do something", async () => {
    // Your test here
  })
})

Source: packages/core/src/tests/TESTING.md:3-19

Identify dependencies and mock them accordingly (e.g., external APIs, database, file system).
Use the appropriate template for testing evaluators, workflow functions, database operations, or AI API calls.

Running Tests

Command	Description
`bun run test`	Run all unit tests.
`bun run test:smoke`	Run fast smoke tests.
`bun run test:gate`	Run comprehensive gate tests.
`bun run test:integration`	Run only integration tests.
`bunx vitest run <path_to_test>`	Run a specific test file.

Sources: README.md:57-58, tests/README.md:21-29

Persistence Layer

The persistence layer is designed with clean architecture principles, supporting multiple backends like Supabase and an in-memory store for testing.

Factory Pattern

To instantiate the persistence layer, use the createPersistence factory. This allows for dependency injection and easy swapping of backends.

// For production with Supabase
import { createPersistence } from '@together/adapter-supabase'
const persistence = createPersistence({ backend: 'supabase' })

// For testing with in-memory storage
const persistence = createPersistence({ backend: 'memory' })

This pattern is also used for running the application without a database connection by setting the USE_MOCK_PERSISTENCE=true environment variable.

Sources: packages/adapter-supabase/REFACTORING.md:10-18, README.md:175-176

Database Schema

The core database schema tracks workflows, their versions, invocations, and the evolutionary history.

The diagram below outlines the main tables and their relationships.

Generating architecture diagram...

Source: packages/core/src/utils/persistence/README.md:25-72

Code Style

The project uses Biome for code formatting and linting. Configuration is located in biome.json.

To format all files: bun run format
To check for formatting issues: bun run format:check

Path aliases like @core, @shared, and @lucky/tools should be used instead of relative paths to improve readability and maintainability.

Sources: README.md:85-89, biome.json

Developer Guide

Project Structure

Installation and Setup

Prerequisites

Initial Setup

Configuration

Common Tasks

Tool Architecture

The defineTool Pattern

Tool Auto-Discovery

Tool Registration Flow

Testing

Test Directory Structure

Shared Test Infrastructure

Writing Tests: Quick Start

Running Tests

Persistence Layer

Factory Pattern

Database Schema

Code Style

Developer Guide

Project Structure

Installation and Setup

Prerequisites

Initial Setup

Configuration

Common Tasks

Tool Architecture

The defineTool Pattern

Tool Auto-Discovery

Tool Registration Flow

Testing

Test Directory Structure

Shared Test Infrastructure

Writing Tests: Quick Start

Running Tests

Persistence Layer

Factory Pattern

Database Schema

Code Style

The `defineTool` Pattern

The `defineTool` Pattern