Database Migrations & Schema

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• apps/web/src/app/(app)/dashboard/page.tsx
• services/api/scripts/seed_agents_and_datasets.py
• services/api/scripts/seed_skills.py
• services/api/flow/interfaces/http/routes/golden_sets.py
• services/api/flow/interfaces/http/routes/knowledge.py

The Flow platform utilizes PostgreSQL with the pgvector extension as its primary relational and vector database. The system manages complex relationships between users, workspaces, autonomous agents, and their associated knowledge bases.

Database versioning is handled through Alembic migrations, which run automatically on API startup. As of the current version, the system has progressed through 12 specific migrations, ranging from the initial schema to the implementation of agent schedules.

Sources: README.md:2, 146-147, 185-187

Data Model Overview

The core data model is structured to support multi-tenancy through workspaces, versioned agent configurations (genomes), and a retrieval-augmented generation (RAG) pipeline.

Entity Relationship Diagram

The following diagram illustrates the primary relationships between the core entities in the Flow database.

Core Tables & Schema

The following table summarizes the primary database tables and their functional roles within the application.

Table	Primary Columns	Description
users	id, email, password_hash	Stores authentication credentials and user profiles.
workspaces	id, name	The primary isolation unit for multi-tenancy.
agents	id, workspace_id, name, template, config	Defines autonomous agent entities and their active configurations.
executions	id, agent_id, status, user_message	Logs individual runs of agents, including status and messages.
knowledge_sources	id, workspace_id, title, ingest_status	Metadata for uploaded documents or crawled URLs.
knowledge_chunks	id, source_id, content, embedding	Vectorized segments of text used for semantic search via pgvector.
agent_skills	agent_id, name, version, content_md, active	Stores specialized Anthropic-style skills with YAML/XML formatting.
golden_sets	id, workspace_id, name	Curated test datasets for agent evaluation.

Sources: apps/web/src/app/(app)/dashboard/page.tsx:75-131, services/api/flow/interfaces/http/routes/golden_sets.py:270-275, services/api/scripts/seed_skills.py:808-816

Knowledge & Vector Storage

Flow utilizes a hierarchical structure for knowledge ingestion. When a document is uploaded or a URL is crawled, it is transformed into a knowledge_source and then subdivided into knowledge_chunks.

Ingestion Logic

1. Source Creation: A record is created in knowledge_sources with an ingest_status of "processing".
2. Chunking: Documents are split into segments.
3. Embedding: Each chunk is processed via an embedding model (requires OPENAI_API_KEY) and stored in the embedding column of the knowledge_chunks table using pgvector's vector type.

Agent Versioning (Genome)

Agent configurations are treated as versioned "genomes". Every change to an agent's system_prompt, llm_config, tools, or skills snapshots a new version.

Version Lifecycle

• CANDIDATE: A version generated by the Prompt Rewriter or manual edits pending approval.
• ACTIVE: The current version used for all executions. Only one version is active per agent.
• ARCHIVED: Previous versions retained for history and A/B testing.

Sources: README.md:104-115, services/api/scripts/seed_skills.py:793-802

Evaluation & Golden Sets

The schema supports continuous improvement through a dedicated evaluation system.

• golden_sets: Collections of test cases (input/expected output).
• golden_items: Individual test pairs within a set.
• golden_results: Records the performance of a specific agent version against a golden item, including a score (0.0–1.0) and grading_rationale provided by an LLM judge.

Sources: services/api/flow/interfaces/http/routes/golden_sets.py:270-305, README.md:135-139

Seeding & Initialization

The repository provides specialized scripts to populate the schema with canonical data for various domains.

• seed_agents_and_datasets.py: Initializes the 7 canonical agents (e.g., Research Analyst, Code Review Agent) and their associated evaluation sets.
• seed_skills.py: Idempotently inserts specialized skills into the agent_skills table. It uses a versioning check (MAX(version)) to ensure skills are updated rather than duplicated.

# From seed_skills.py: Checking for existing versions before seeding
row = await pool.fetchrow(
    "SELECT COALESCE(MAX(version), 0) AS max_v FROM agent_skills WHERE agent_id=$1 AND name=$2",
    agent_id,
    name,
)
max_v = row["max_v"] if row else 0

if max_v > 0:
    # Already seeded — skip (idempotent)
    continue

Sources: services/api/scripts/seed_agents_and_datasets.py:24-32, services/api/scripts/seed_skills.py:790-805

Conclusion

The database architecture of Flow is designed for robust multi-tenancy and iterative agent development. By combining traditional relational tables for management with pgvector for semantic search and versioned genomes for agent configuration, the schema provides a foundation for the platform's self-improvement loops and RAG capabilities.