Behavioral Self-Improvement

Relevant source files

The following files were used as context for generating this wiki page:

• packages/core/src/prompts/improveWorkflow.p.ts
• packages/core/src/prompts/workflowAnalysisPrompts.ts
• packages/core/README.md
• apps/web/requirements.md
• packages/core/src/prompts/evaluator/root-cause/rca.ts
• apps/web/src/lib/workflow_setup_examples/advanced-examples/meta-workflow.json
• apps/web/src/lib/workflow_setup_examples/my-workflow-judge.json
• README.md

Behavioral Self-Improvement is a core capability of the lucky framework, designed to create an autonomous workflow engine where LLM-powered agents (nodes) can iteratively learn, mutate, and compose themselves into more efficient and successful graphs. This system enables workflows to evolve by analyzing their own performance and applying targeted modifications, treating the workflow definitions themselves as evolvable data structures.

The system employs two primary strategies for optimization:

1. Cultural Learning (Iterative Improvement): A single workflow is analyzed after execution to identify bottlenecks and areas for improvement. Targeted, incremental changes are then applied and validated.
2. Genetic Programming: A population-based approach where multiple workflow "genomes" are evolved over generations using operators like crossover and mutation to discover optimal collaboration patterns.

This document focuses on the iterative, analysis-driven "Cultural Learning" aspect of self-improvement.

Sources: apps/web/requirements.md, packages/core/README.md:28-31, 50-53, README.md:49-52

Architecture and Core Concepts

The self-improvement capability is encapsulated within the Evolution System, one of the five core architectural layers of the platform. It is responsible for all autonomous workflow optimization.

Sources: packages/core/README.md:25, 47-49

The following diagram illustrates the position of the Evolution System within the overall architecture.

The Improvement Loop

The iterative improvement process follows a distinct loop where a workflow's execution is analyzed, and changes are proposed and applied. This is often referred to as the "Cultural Evolution" mode. The core goal is to have a "critic agent" analyze execution traces and suggest mutations, which are then evaluated to promote better-performing workflow versions.

Sources: apps/web/requirements.md:37-40, packages/core/README.md:63-69

This loop can be visualized as follows:

Performance Analysis

After a workflow run, its performance is evaluated based on a set of fitness metrics. This data, along with execution transcripts, forms the basis for all improvement decisions.

Sources: packages/core/src/prompts/improveWorkflow.p.ts:10-14

Fitness Metrics

The system uses a multi-dimensional scoring system to calculate a workflow's fitness.

Metric	Description
Score	An overall performance score, typically out of 100.
Total Cost	The total USD cost of the workflow execution.
Total Time	The total time in seconds for the workflow to complete.
Data Accuracy	A measure of the correctness of the workflow's output.

Sources: packages/core/src/prompts/workflowAnalysisPrompts.ts:50-55

Root Cause Analysis (RCA)

A key step in the analysis is performing a Root Cause Analysis to understand why a workflow is not achieving a perfect score. An LLM agent is prompted to act as an expert debugger, analyzing the execution trace to find where the system went wrong. The analysis must be based on evidence from the trace and should not suggest improvements, only identify causes.

The RCA process involves two parts:

1. Systematic Investigation: Analyze the agent trace, identify specific nodes or tool interactions causing issues, and state the severity of each root cause (e.g., fatal, high, medium, low).
2. Assumption Breakdown: Question the original problem statement to uncover hidden assumptions, form hypotheses, and identify key findings based on engineering first principles.

Sources: packages/core/src/prompts/evaluator/root-cause/rca.ts:4-38

Improvement Proposals

Based on the performance analysis, the system generates proposals for improving the workflow. This is handled by LLM agents acting in specific roles, such as a "Judge" or a "Mechanic Advisor".

Sources: packages/core/srcsrc/prompts/improveWorkflow.p.ts

The "Judge" and "Mechanic" Roles

The improvement process uses specialized LLM agent roles defined by specific system prompts:

• Workflow Optimization Judge: This agent's role is to analyze a workflow's configuration, fitness metrics, and feedback. It decides whether an improvement is needed. If so, it returns a complete, improved WorkflowConfig JSON. If the workflow is already optimal (e.g., fitness score of 100), it returns null. A key instruction for the Judge is to always preserve the memory field of existing nodes.
• Mechanic Advisor: This agent provides advice on the single most impactful change to make. Instead of generating a new workflow, it returns a structured object detailing a specific action. This advice is then used by another system to formalize and apply the change. There are two types of mechanic advisors: one for node-level changes and one for structural changes.

Sources: packages/core/src/prompts/improveWorkflow.p.ts:7-24, 80-97, 185-199

The interaction between these components can be visualized in the following sequence diagram:

Improvement Actions

The "Mechanic Advisor" can suggest one of several actions to modify the workflow.

Action	Description
addNode	Adds a new node to address a specific bottleneck. The advice includes the full JSON for the new node and its predecessors.
removeNode	Removes a redundant or problematic node. The advice specifies the target nodeId.
modifyNode	Modifies an existing node. The advice includes the target nodeId and natural language instructions for the modification.
doNothing	Indicates the workflow is already optimal.
customOperation	Describes a precise, workflow-level change that goes beyond single-node operations.

Sources: packages/core/src/prompts/improveWorkflow.p.ts:84-91

The improvement task instructions guide the LLM to propose changes such as adding, editing, or removing nodes, and restructuring connections for better parallelism or flow.

Sources: packages/core/src/prompts/workflowAnalysisPrompts.ts:57-64

Improvement Strategies and Patterns

The self-improvement system is aware of various established patterns for both workflow structure and prompt engineering, which it can leverage when proposing changes.

Workflow Structure Patterns

The system can recommend and implement various architectural patterns to improve flow and efficiency.

Pattern	Description
LLM-as-a-judge	Adds a node that evaluates and scores outputs from previous nodes.
Multi-step reasoning	Sequences nodes where each builds on the previous one's output.
Validation pipeline	Creates nodes for input checking, main processing, and output verification.
Iterative refinement	Simulates feedback loops through sequential nodes (e.g., generate, critique, improve).
Specialized experts	Assigns different nodes to handle specific aspects like analysis or formatting.
Quality gates	Inserts checkpoint nodes that assess and potentially reroute based on quality.
Fallback patterns	Adds alternative paths triggered when primary nodes fail.
Data transformation pipeline	Chains nodes that progressively modify and enrich data.

Sources: packages/core/src/prompts/workflowAnalysisPrompts.ts:11-23

Prompt Engineering Patterns

The system can also suggest modifications to node prompts based on proven prompt engineering techniques. Examples include:

• Ensemble prompting
• Chain-of-thought
• Role-playing
• Self-reflection
• Structured output (e.g., JSON)
• Few-shot prompting

Sources: packages/core/src/prompts/workflowAnalysisPrompts.ts:28-48

Example Meta-Workflows

The concept of analysis and judgment is so central that it can be implemented as a workflow itself. The repository contains examples of "meta-workflows" designed to analyze other workflows.

Meta-Workflow for Improvement Feedback

The meta-workflow.json file defines a 4-step workflow designed to analyze another workflow's execution and produce actionable improvement feedback.

This diagram shows the structure of the meta-workflow.

This workflow demonstrates the "Multi-step reasoning" and "Specialized experts" patterns, where each node has a distinct role in the analysis process, building upon the previous one's output. The final synthesis-node is specifically prompted to provide "clear, actionable feedback" and "avoid rehashing analysis," focusing entirely on implementable improvements.

Sources: apps/web/src/lib/workflow_setup_examples/advanced-examples/meta-workflow.json

Judge Workflow for SWE-bench

The my-workflow-judge.json provides a more complex example of a workflow structured for detailed analysis, planning, and implementation, likely for a software engineering task. It includes specialized nodes for:

• Requirement Analysis (requirement_analyzer)
• Technical Analysis (technical_analyzer)
• Implementation Planning (implementation_planner)
• File Operations (file_operation_coordinator, file_manager)
• Code Implementation and Validation (code_writer, code_implementer, validator)
• Final Output Generation (patch_generator, solution_validator)

This workflow exemplifies a highly structured, sequential pipeline with specialized agents, demonstrating the "Validation pipeline" and "Data transformation pipeline" patterns.

Sources: apps/web/src/lib/workflow_setup_examples/my-workflow-judge.json

Summary

The Behavioral Self-Improvement system is a sophisticated feature that enables lucky workflows to autonomously evolve. By combining performance metrics, root cause analysis, and LLM-driven "Judge" and "Mechanic" agents, the system can identify weaknesses and apply targeted improvements. It leverages a rich set of predefined workflow and prompt patterns to guide its optimization strategies, moving the project closer to its core goal of a fully autonomous, self-composing workflow engine.

Sources: apps/web/requirements.md:5-7, packages/core/README.md:50-53